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ABSTRACT 

The use of analyses of achievement test data, 
particularly longitudinal data, in a wide range of court cases 
bearing upon educational policy and practice Is examined, with an 
emphasis on assessments of the educational progress of students. The 
main purpose of the project Is to show that assessments of 
educational progress (I.e., student learning) are Integral to many 
court cases and related educational policy issues. Project 
participants were educational researchers and specialists in 
quantitative methods. The major research questions addressed the 
types of questions about student progress that arise in educational 
court cases, the types of data and statistical summaries that are 
presented in court cases, courts' responses to the empirical evidence 
on student progress, and presentations that prove most successful. 
The representation of academic progress by student growth in 
achievement, the definition and representation of expected growth, 
and data and methods used in practice to assess student progress are 
discussed. Specific educational court cases involving the measurement 
of student progress are presented covering the evaluation of teacher 
performance, racial discrimination, mandated desegregation, and 
student classificatioii. Reanalyses of the evidence presented in two 
cases are also described. Finally, suggestions are made for improved 
data collection, data management, and statistical analysis. A ei-item 
list o:f references, an annotated bibliography of additional sources, 
and two computer programs for analyzing student progress are 
included. (TJH) 
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EXECUTIVE SUMMARY 



This research exa ines the use of analyses of achievement test 
data in a wide rar je of court cases bearing upon educational 
policy and practice. The particular focus of the project was on 
assessments of the educational progress of students. The 
participants in this project are educational researchers and 
specialists in quantitative methods. The aim was to study the use 
of empirical evidence in the court cases and to apply recent 
advances in the conceptualization and analysis of longitudinal 
data. A main p irpose of the project is to show that assessments 
of educational progress (i.e., student learning) are integral to 
many court cases and related educational policy issues. 
Subsequent to identifying questions about student learning in a 
variety of court cases, the project sought to critique current 
practices and to offer prescriptions for improvement. 

The major questions addressed in the project are: 

* What kinds of questions about student progress in 
achievement arise in the educational court cases? 

* What types of data and statistical summaries are presented 
in the court cases? 

* What are the court's responses to the empirical evidence 
on stu-ient progress and what kinds of presentations would 
be most effective? 

Although many important questions about student progress arise in 
the court cases, in none of the cases reviewed was a fully sound 
or satisfactory analysis of progress located. In many of the 
cases the student achievement data were seriously deficient, and 
in others the statistical analyses were weak or misleading 
(reflecting also the current state of practice in educational and 
behavioral science research) . Recommendations for improved data 
collection designs are given, and improved statistical methods for 
the assessment of student progress are described which are also 
implemented in a set of computer programs. 
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USE OF ACHIEVEMENT MEASURES IN JUDICIAL DECISIONS: 
ASSESSMENT OF STUDENT ACADEMIC PROGRESS 



SECTION 0 
PURPOSE AND OVERVIEW 

This research project exeuulnee the use of an&lyses of 
achievement test data in a wide range of court cases bearing upon 
educational policy and practice. The project seeks to apply 
recent advances in the conceptualization and analysis of 
longitudinal data; thus, the focus is on the assessment of the 
educational progress of students. The work concentrated on uses 
of test scores for assessing student progress in court cases ^.nd 
on the presentation of improved and more appropriate methods and 
designs. A main purpose of the report is to show that 
assessments of educational progress (learning) are integral to 
many key judicial and related policy issues. Subsequent to 
identifying questions about student learning in a variety of 
court cases, the project sought to critique current practices and 
to offer prescriptions for improvement. 

The major questions addressed in the project are: 

0 What kinds of questions about student progress in 

achievement arise in the educational court cases? 
0 What types of data and st?tistical summaries are 

presented in the court cases? 

0-1 
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0 What are the court's rtsponses to the empirical 
evidence on student progress and what kinds of 
presentations would be most effective? 

The participants in this project are educational researchers 
and specialists in quantitative methods. Thus, the emphasis of 
the work was on the kinds of empirical evidence that has or could 
be presented to the court, rather than on technical analysis of 
legal issues. This project is located in an interesting niche 
where interests and concerns of research statisticians, 
educational researchers, legal experts and education policy 
makers Intersect. 

The report is divided into four main sections. In the first 
section we (1) present the basic ideas of empirical assessment of 
student progress, (11) outline some of the issues involved in 
definitions of **expected growth" or ••normal educational 
progress^^, and (ill) conclude with descriptions of different 
types of test data and of the different types of statistical 
analyses used for the assessment of student progress. The second 
section reviews the different types of court cases examined in 
the first phase of this research. We identified four types 
(sometimes overlapping) of court cases in which assessments cf 
student progress figure prominently: evaluation of teacher 
performance, racial discrimination cases, evaluation of school 
desegregation programs and student classification cases. The 
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third ••ctlon of the report focuses on methodfit for statistical 
analyses of student progress through the critique and refinalysis 
of two court cases which both raised basic issues about student 
progress and also whose data was obtainable for reanalysis. The 
fourth section attempts to explain what we feel are the valuable 
lessons learned from this research project. In particular, we 
offer recommendations for kinds of data and statistical analyses 
that would serve to provide useful evidence on the questions 
about student academic progress raised in many of these court 
cases. And second we offer a description of three kinds of 
important, but not fully understood, technical problems that 
would serve as a useful guide for future methodological research. 
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SECTION 1 

ISSUES IK THE MEASUREMENT OF STUDENT PROGRESS 



This section first presents the ide&s and the data structure 
of individual histories of achievement and their use in 
statistical assessments of student progress. Next, the possible 
definitions of "expected growth" or "normal educational progress" 
are examined. Third, we consider the various forms and 
transformations of test scores that might be or have beon used in 
assessments of student progress, and discuss the statistical 
methods used to summarize test scores. The aim is both to show 
the diversity of various methods that arise in the presentation 
of statistical arguments in relevant court cases and to establish 
a framework for thinking about useful methods for assessing 
student progress. 



SECTION 1.1: Representing Academic Progress by Student Gicwth In 
Achievement* 

School districts regularly assess student performance using 
group administered achievement tests. Such testing represents a 
large Investment of money and time for the schools, for 
administrators, for teachers, and for students. Yet relatively 
little use Is made of the accumulated test data. In particular, 
test results are presented In a way that describe only the 
current status of students; the data are presented as a static 
"snapshot" of achievement without any link to prior levelB of 
performance. Even the management of test data reflect these 
limitations. Whether the test results be stored as hard copy or 
electronically, achievement data are organized as separate yearly 
files, which may be located on separate physical devices and even 
In separate geographical locations. 

A key to the Improved use of achievement test data Is to use 
performance on repeated tests to describe student learning. A 
student's score at a single point In time cannot be used to 
measure learning: collecting together scores from previous 
testing Is necessary for the analysis of student progress. A 
student's "cumulative folder" Is organized In this manner, but 
these are rarely stored electronically nor uniformly maintained. 

It Is regrettable that achievement results tend to be 
presented within a static framework— the data for a given group 
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of students are shown without any ties to their earlier 
performance and without any reference to the instructional 
program. Much important information is lost when test results 
are presented in a way that only informs about current behavior. 
A major obstacle to the improvement of the use of achievement 
test data is that school administrators and researchers or expert 
witnesses working with the school administrators do not have any 
practical models or methods to follow in bridging the gap between 
static snapshots of achievement data and assessment of student 
progress. Although questions about student progress or student 
learning are frequently seen in the court cases, these questions 
are rarely addressed adequately. 

For improved use of achievement test data, the key is to use 
performance on the repeated tests to describe student learning 
processes. A basic belief underpinning the work of this project 
is that statistical analyses of individual achievement histories 
can be highly informative for description of student learning and 
for identification of strengths a*id weaknesses of instructional 
programs. By bringing together historical data from several 
years for individual students, useful indices for student 
progress can be calculated. This approach differs in two 
significant ways from methods currently used by practitioners and 
researchers. First, the student is the basic unit of observation 
and analysis. Rather than focusing on average scores which will 
obscure Important individual differences, we begin with an 
analysis of each individual's record. Second, the emphasis is on 
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nultl-year trends rather than pre-post year to year changes, as 
information about progress or growth Is extremely limited when 
only two points in time are available. 

Though the notion of individual growth curves is not new in 
the study of learning and human development, educational 
researchers and policy analysts have given littla attention to 
this Important topic. A primary goal of this study is to 
advocate the creation and use of student histories of 
achievement, and to provide guidance for analyzing this 
information in the context of court cases. 

Statistical Models f or Individual Growths Psychological 
learning theory and biological growth research provide a variety 
of complex models of individual growth such as polynomial growth 
curves, logistic growth curves, and simplex models (see Rogosa 
and Willett, 1985, for an exposition of these models) . The 
simplest models, and the one we concentrate upon for practical 
application, is a straight line growth model which assumes a 
constant rate of learning for each individual. Thus, the rate of 
Improvement provides a simple index for the learning of each 
individual. The straight line growth model is useful for 
heuristic reasons because of its simplicity and also serves as a 
useful approximation to actual growth processes. Moreover, the 
common use of grade equivalent scores for achievement test data 
which implicitly assumes a constant rate of change model with 
unit slope makes exposition of these kinds of models practicable 
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to non-experts. Finally, the rather small number of observations 
typically available on each student (perhaps four) justify 
fitting models only as simple as the straight line growth curve 
model . 

Figure 1 provides four examples of possible data for each 
individual to illustrate the data and the straight line growth 
model. For example, student A exhibits a rapid rate increase 
from a relatively low initial status. Student B starts out 
somewhat higher than sii:udent A and grows at a similar rate. 
Student C exhibits slov growth from a relatively low initial 
achievement level and finally student D shows a high level of 
achievement but no growth due to the ceiling effect of the test 
used. 



Insert Figure 1 here 
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Section I.l 




Figure 1. Contrastive growth trajectories for four individual students: (a) low entry, 
fast steady growth; (b) average entry, steady growth; (c)low entry, slow erratic progress 
(d) high entry, ceiling effect. 



SECTION 1.2: Definition and Representation of Expected Growth 

Comparison with a standard or criterion Is a problem common 
to many uses of test data. Even with the static snapshot of 
achievement at a certain point In time, the Interpretation of 
Individual or group-averaged scores often rests upon the 
individual's standing relative to a national or local group 
(i.e., calculations of percentile rank scores) or the comparison 
of one group average with another (i.e., comparison of schools or 
district means with each other or with the state average.) 
Certainly, such concerns do not evaporate when indices of 
progress over time are at issue rather than snapshots at a single 
time. What is often desired after some measure of student 
progress has been calculated and or aggregated, is a comparative 
assessment of that index vis-a-vis some expectation. This comon 
goal is a rationale for the preponderance of concern with normal 
educational progress or adequate progress that is seen in many of 
the court decisions. Determination of such standards or 
criteria, even with a sound statistical approach for assessing 
progress, is not unambiguous. The question, "what constitutes 
normal educational progress?" cannot be answered by technical 
considerations alone. Even if norms of measures of progress are 
available, it is unclear whether such criteria should be 
established unconditionally or conditionally. By 
"unconditionally" we mean a norm of progress that does not take 
into account student background, student initial status, or other 
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possible Influsnces on student progress for different types of 
students; an "unconditional" standard nay not be the fairest or 
most persuasive In a judicial proceeding. Conditional criteria 
respond more to questions such as, "Nhat Is nonal educational 
progress for a certain kind of student (e.g., ethnic minority)?" 
or, "Is normal educational progress different for students who., 
say J entered high school with very different levels of 
achievement?" or, "Is It adequate In evaluating a school 
desegregation program to determine that minority students have 
the same rate of progress as majority students even though a 
large preexisting gap between the level of achievement of 
minority and non-mlnorlty students wapi present before 
desegregation remedies were Implemented?" As there Is no natural 
metric, save that established artificially by grade equivalent 
scores, this Important question has no simple or general answer. 

But a crucial part of the use of analyses of student progess 
Is a comparison with "expected" or "normal" growth. The beot 
general advice Is that In each specific application, explicit 
definitions of "adequate" progress should be made and deliberated 
among the parties Involved. The two definitions of "normal 
educational progress" that are commonly used In court cases are 
based on either the grade equivalent score (Sections 1.3.1.4, 
1.3.2.3) or the percentile rank score (Section 1.3.1.3). An 
Improvement of 1.0 GE unit per school year Is the most widely 
used criterion for normal progress. Alternatively, maintenance 
of percentile rank score over successive years Is taken as 
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"nomal*' growth (Section 1.3.2.3) In the sense that the student 
is moving along with the distribution of scores over time. In 
practice, these two criteria may frequently yield different 
conclusions for the same test scores. 
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Sactlon 1.3: Data and methods used In practice to assess student 
progress. 

The purpose of this section Is to define and distinguish 
among the collection of data transformations of test scores and 
data analysis methods for the assessment of student progress that 
arise In the court cases. The first part of this section 
distinguishes among the various commonly used measures of 
scholastic achievement; that is, the various scores used to 
represent status of an examinee on a certain test or task. The 
most obvious of these is the raw score; more common are many 
complex transformations of the raw score. The second part of the 
section describes statistical analyses using one or more of the 
kinds of test scores for the assessment of scholastic progress. 
In the simplest case, there may be two measures, say, a year 
apart, available on each student and various statistical methods 
may then be applied to draw conclusions about the growth of 
groups of students and also to compare their growth with some 
sort of norm. 

1.3.1. Different forms of test data. 

This section describes raw scores and their common 
transformations: grade equivalent units, percentile ranks, normal 
curve equivalents (NCE) , and scale scores. 
1.3.1.1 Raw Scores. 

The raw score is the simplest and most basic Information on 
the examinee p'srformance, being .simply the total number of 
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correct rasponses on tlie test or aubtast. The raw score la the 
basis for all derived scores In that any score Is based on a 
transformation of the raw score. The advantage of the raw score 
(or Its equivalent, percent correct) Is the simplicity of 
Interpretation; a student taking the same form of the same test 
In two successive years may In the second year get seven more 
Items right than she did In the first year. The seven Item 
Increase provides a simple Idea of the student's Improvement In 
scholastic Improvement. For the static assessment of achievement 
level at a single point In time comparison of the raw scorci to 
the total number of Items— which yields the percent correct— Is 
obviously more Interpretable In Isolation than the raw score 
Itself. 

1.3.1.2 Normal Curve Equivalents (NCT) . nce»s are a simple 
transformation of the raw score which serves In the attempt to 
give a readily Interpretable static assessment of Individual 
status for one point In time. If the raw score Is denoted by X, 
the common form of the normal curve equivalent score Is given by 

" ^ ) (21.06) + 50 

where "21.06 was d.^rlved by dividing the distance from the mean 
to the 99th percentile. . .by the same distance measured In terms 
of normal curve standard deviation units." (Tallmadge, G.K. and 
Wood, C.W. (1976) User's Guide! ESEA T itle 1 Evaluation and 
Reporting System . Mountain Vlew:RMC Research Corporation.) That 
is, 21.06 - (99 - 50)/2.3267 - 49/2.3267. In the expression, M 
and S are the "estimated" national mean and standard deviation 
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(Tallmaige and Wood p. 16) Thus, the NCE Is just a simple linear 
transformation of the raw scores. In particular, for a single 
point in time, the correlation r}|CE,RAW*^*^* 

In the definition of the NCE, the anchor points of 
percentiles 1 and 99 are arbitrary. For NCE*s the multiplier 
(21.06) corresponds to anchor points of 1, 50, and 99. Define a 
as the area in one tail, so above a ■ .01. In general the anchor 
points are reflected in the multiplicative constant, ({1- 
2a)/2}(100)/Za, where is the upper (1-a) fractile in the 
N(0,1) distribution. So for example, if instead we want the 
transformed scale to be anchored at 5, 50, and 95 (a - .05), the 
scaling coefficient would be 100{(1 - .10)/2)/1.64 - 45/1.64 « 
27.44 rather than 21.06. 

Although Raw scores and NCE's have a simple 
scale/translation relation at any single slice in time, use of 
the NCE metric for longitudinal data involves serious 
complexities. In particular, tht^' norming constants (M and S) 
presumably will change at different points in time. Therefore 
the correspondence between raw score and NCE will not hold for 
assessments over time. One example of the use of NCE scores in a 
court case is described in section 2.4.2.1 (naacp v. State of 
Georgia ^ where improvement in NCE scores in successive years for 
individual counties was used as evidence that student 
classification practices were not harmful. 
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1.3.1.3 Percentile ranks. 

Percentile ranks, one of the most common test score 
reporting metrics, simply indicate the percentage of raw scores 
In a norm group that fall below a given student's raw score. 
Thus, percentile ranks are a transformation of the raw score that 
Is defined by the empirical cumulative distribution function of 
the norming sample. The form of this transformation Is not 
linear but It Is monltonlcally Increasing with raw score. As an 
Indication of relative standing to the norm group, the percentile 
rank has an obvious but limited Interpretation for measuring 
achievement at a single point In time. For longitudinal 
assessment, the percentile rank score does not allow a simple 
Interpretation of how much more a student Icnows at time 2 than at 
time 1, but percentile rank might be used to Indicate whether the 
student Is maintaining his or her standing relative to the norm 
group over time. As a percentile rank score cannot be used as an 
Interval scale, more complex longitudinal statistical analyses of 
percentile ranks are not attractive. 

1.3.1.4 Grade Equivalent Scores 

Grade Equivalent scores (GE) are perhaps the most common and 
most abused scale for reporting results of standardized 
achievement tests. A grade equivalent represents the grade and 
month in school of students in the norm group whose test 
performance is theoretically equivalent to the test performance 
of a given student. A vast psychometric literature exists to 
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diocouraga the ovarinterpretation of grade equivalent scores, and 
this literature serves a useful purpose. However, in urging that 
6E*8 not be over interpreted, sometimes the useful aspects of GE 
scores are not recognized* 

The best way to think of GE scores is that like all other 
derived scoies GE»s are a transformation of the student's raw 
score on the test. However, the form of this transformation is 
usually more complex and less explicitly defined than for any of 
the other kinds of derived scores. A very large advantage of GE 
scores is their simplicity, both in reporting status in a single 
point in time and for the implicit characteristic of GE's in 
representing change over time. The simplicity of a unit increase 
per year of school has a compelling intuitive basis, if not 
always a sound psychometric basis. 

Grade Equivalent scores are perhaps the most widely used 
metric for test data in court cases; for example, GE's are seen 
to be readily acceptable by the courts in the cases Tattnall 
County (Section 2.2.3) and naacp v. the State of Georgia (Section 
2.4.2.1) and Scheelhaaae v. Woodbury (Section 2.1.1). If GE 
scores are to be deemed inadequate for technical reasons, then it 
may be necessary to have replacements with equally simple 
interpretations. Clearly additional complexity no matter its 
technical soundness is not necessarily efficacious in complex 
judicial proceedings. 

1.3.1.5 Grade Equivalent Norms for Selected Tests. 
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order to investigate the relationship between Grade 
Equivalent scores (GE) and raw scores (RS) , we constructed plots 
of the GE's versus raw scores from norms tables for selected 
subtests. In general, we used a reading (or verbal aphievement) 
score and a mathematics achievement score from each test battery. 
Table 1 shows the test batteries used, the specific level and 
form, and the recommended grade range for each. The Iowa Tests 
of Basic Skills (ITBS) were published by Houghton Mifflin. All 
the other tests were published by CTB/McGraw-Hill. These 
particular test batteries were chosen because of th^ir wide use 
and the availability of norms tables. 



Insert Table 1 here 



Table 2 shows the maximum and minimum GE's and the raw score 
or range of raw scores that corresponds to those GE*s for each 
test used. The names of the test batteries are abbreviated: CAT 
for California Achievement Tests, CTBS for Comprehensive Tests of 
Basic Skills and ITBS for Iowa Tests of Basic Skills. Each test 
battery abbreviation is followed by a slash and the designation 
for its level and form. Thus CAT/19D stands for California 
Achievement Tests, level 19, Form D. It can be seen that there 
is considerable variability in the correspondence between GE's 
and raw scores at the extremes of the scales. Some tests have 
one-to-one correspondences between GE and raw scores while others 
show a broad range of raw scores corresponding to the extreme 
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TABLE 1 





TESTS ANALYZED 


IN FIGURES 


1-16 


TEST BATTERY 




TQm 


recqukended 


CAT 


19 


C 


9 . 6-12 » 9 


CAT 


19 


D 


9.6-12.9 


CAT 


5 


A 


9-12 


CAT 


3 


A 


4-6 


CAT 


2 


A 


2-4 


ITBS 


11 


6 


5 


ITBS 


14 


6 


9 


CTBS 


1 


S 


2.5-4.9 
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6E*8. Note particularly that the naxlmum 6E on several tests can 
be achieved by answering less than 80% of the Items correctly. 



Insert Table 2 here 



Figures 1-16 show the plots of grade equivalent versus raw 
score. The functional form of the GE-RS correspondence varies 
depending on the test. The CAT tests designed for use In the 
high school grades (9-12) generally show some celling and floor 
effects, as noted In Table 2 and Figures 1-6. In the 
Intermediate range the scales show approximately a stralght-llne 
trend for the reading tests and a slight s-shape on the math 
tests. However even on the math tests, a stralght-llne might fit 
reasonably well for the middle range of scores since the 
curvature Is slight. 



Insert Figures 1-16 here 



The CAT tests designed for the elementary grades (CAT/2 A and 
CAT/3A) show a different pattern. There is some floor effect on 
each test (Figures 7-10) . However, these tests show considerable 
curvature in the rriationship between GE and raw scores with the 
curve taking a steep upward turn at roughly 3/4 of the maximum 
raw score. No culling effect is exhibited. 

The GE and raw scores scales on the ITBS subtests (Figure 
11-14) show roughly a linear relation throughout the entire range 
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RAW SCORCi 


RANG£S rOR TriSTS 


Iff TAoJuEi 1 




TEST 


MIN GE 


RAW SCORE 


MAX GE 


RAW SCORE 


CAT/19C natn 


2 • 0 


0-7 


12 • 5 


54-85 


CAT/ 19 D Math 


2 • 0 


0-7 


12 .5 


53-85 


CAT/5A Hatn 


0. 6 


0-3 


13 . 6 


77-98 


CAT/ 3 A Hatn 


0.6 


A *1 A 

0-10 


11. 0 


*1 A A 

108 


CAT/ 2 A Math 


0* 6 


0-28 


6.3 


117 


iToS/ii|0 Hath 


!• > 
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A A 

9 . 0 


A A 

42 


1 1 x5b/ 14 , D Ha v^n 


J • 0 
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12 • 9 
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CTBS/15 Hath 


1 A 
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0-12 
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98 




2 • U 


U— 7 


12 • 9 
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CAT/ 19 D Reading 
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12.9 


46-70 


CAT/ 5 A Reading 
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0-6 


13.6 
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CAT/3A Reading 
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0-10 


1^.9 


82 


CAT/2A Reading 
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0-23 


8.4 


85 


ITBS/11,6 Verbal 


1.0 


0 


9»2 


43 


ITBS/14,6 Verbal 


2.3 
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12.9 


47 


CTBS/15 Reading 


1.0 


0-9 


9.9 


83-85 
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of the scales with only a minor floor effect on the Verbal tests. 
The raw scores an*l 6E*s exhibit a 1-1 correspondence for the most 
part, with two raw scores occasionally mapping into a given GE. 

The 6E-raw scores relation on the CTBS/IS (Figures 15 & 16) 
is similar to that for the CAT/2A and CAT/3 A, the other 
elementary grade products from CTB/Mc6raw-Hill. There is some 
floor effect. The curve then climbs slowly, approximately 
linearly, until it takes a steep upward climb at about 3/4 the 
maximum raw score. From that point on the curve climbs rapidly 
to the maximum point of the GE and raw scores scales. 

1.3.1.6 Scale Scores. 

Scale scores are the most recent and most sophisticated 
transformation of raw scores widely reported for standardized 
tests. These scores hold considerable progress in improving some 
of the undesirable features of other transformations of raw 
scores. The aim of scale scores is to provide a single equal 
interval scale of scores across all grades for use with all 
levels of a standardized test, such as the California Achievement 
Test (CAT) . A key characteristic of scale scores is that they 
aim to be independant of test level or form, grade or time of 
year of testing. Also, the scale scores are intended to form an 
interval scale, that the difference of any two successive scores 
on the scale is always the same in terms of scholastic 
achievement. Thus, test publishers can claim the following 
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ralevant properties of test scores (taken from pg. 54 of the Test 
Coordinators Handbook of the California Achievenent Tests for 
Forms C and D) : 

1. Scale scores can be added, sxibtraoted, and averaged across 
test levels. Such computations make it possible to directly 
compare individuals, classes, schools, or entire districts. 

2. Scale scores facilitate longitudinal studies of achievement 
growth over any period of time for an individual or group. For 
example, scale scoreb can be used to record a students progress 
from the beginning of school through Grade 12. 

So, for purposes of assessing student progress, scale scores 
hold the most promise and could well overcome objections used 
with raw scores or grade equivalents in assessment of student 
progress. However, among the many court cases that were 
reviewed, only in some of the most recent San Francisco 
desegregation analyses (Section 2.3.2) were scale scores used; 
this is likely to be because test developers have only recently 
begun reporting such scores routinely. For example, CTBS tests 
were reported in scale score units starting in 1980. We would 
anticipate more use of scale scores in the future, and this 
should facilitate more useful analyses of student progress. 

1.3.2. Methods for Assessing student Progress. 

Methods for assessing learning or progress in student 
achievement have been a major methodological issue in much of the 
educational and social science literature over the past 50 years. 
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Rogosa, Brandt, and Zlmowskl (1982) and Rogosa and Wlllatt (1985) 
critique this literature. Sone of these methods and their 
associated controversies appear in many of the court cases that 
were studied in this project. Almost any of the types or 
transformations of test scores described in section 1.3.1 can be 
used with the possible methods. However, some of these methods 
have much better or attractive properties with certain kinds of 
test scores than with others. This crossing of method and type 
of test scores produces a panoply of possible forms in which 
student progress might be represented. The purpose of this 
section is to describe some of these methods along with their use 
in certain court cases. 



1.3.2.1 Gain Scores 

The simplest, but often most controversial, method for 
assessing progress is simply to compute a gain or average gain 
between an earlier and later time for a certain test score 
metric. A gain score in raw score units simply relates how many 
more items the stude^ c (or an average of students) were able to 
achieve at time two compared to time one. The grade equivalent 
score has a particularly simple interpretation and natural 
linkage to the calculation of student gains. Grade equivalents 
attempt to provide an interval scale metric for calculating 
student gains expressed in terms of years of schooling. In 
addition, grade equivalents provide an implicit definition of 
expected gain in terms of a GE unit per school year representing 
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some sort of average progress. Another obvious and perhaps 
better use of slnple gains would be in terms of the scale scores 
(see section 1.3.1.5), which may provide a psychometricly more 
acceptable method for gauging student progress. 

Most important, is that gain scores are limited to 
assessments of progress over two points in time— a limitation 
which ca/i be extremely consequential. Assessments of progress 
may often require much more data than just two points in time on 
each student, but the difference score cannot make good use of 
this limited amount of data. When more extensive data are 
available, the difference score will discard important 
information about the form of growth or about nonconstant rate of 
change. 

Examples of the use of gain scores abound in the court 
cases. For example, in the case NAACP v. state of Georgia 
(Section 2.4.2.1) the defendant school districts express id their 
evidence of student progretis in terms of gains over a school year 
both in units of normal cu2rve equivalents and in units of grade 
equivalents. In Scheelhaat te v. Woodbury (sections 2.1.1 and 3.2) 
student progress was summarized by different scores of grade 
equivalents taken at yearly intervals such as between sixth and 
seventh grade. 

Although the basic notion of looking at improvement from one 
grade to the next via a difference score is simple and 
compelling, it is easy to see that the variety of forms in which 
test scores could be reported could yield rather different 
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results from the same basic data. Possible contradictions can be 
illustrated through use of the plots of raw score versus Grade 
Equivalent scores accompanying section 1.3.1.4. The fact that 
the transformation of raw scores into Grade Equivalents is non- 
linear could mean that a sizeable increase in raw score units 
might translate into a very small Grade Equivalent increase in, 
say, the higher percentage correct part of the scale. Yet, on 
the other hand, for the low scoring students a small increase in 
number correct on the test can translate into a large apparent 
gain in Grade Equivalent units. Even less attractive in th«s 
assessment of progress by difference scores is the use of normal 
curve equivalents; because these scores are independently 
referenced to a target population at each separate time point, 
the comparability of the raw score units from which they are 
derived at two different times is destroyed. 

1.3.2.2. Regression Adjustments. 

Various forms of multiple regression procedures are employed 
in the court oases with two broad purposes: (1) prediction of 
later achievement with special attention to the importance of 
demographic or racial variables in the prediction and (2) 
regression adjustments for initial differences on achievement or 
demographic characteristics with the purpose of comparing change 
across initially dissimilar groups. 
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Prediction ft<yuatlon8 . A typical example of the first type 
of multiple regression appears in NAACt^ v. Georgia (section 
2.4.1.1). In the regression analysis the outcome variable was 
tenth grade achievement test score in either reading or math and 
the predictor variables included prior achievement level, racial 
identification of the student, status in free/reduced lunch 
program (an SES indicator) , student retention history, student 
absences, and remedial courses taken in mathematics and language 
arts. Basically, out of a pool of possible predictors, the 
predictors which show the highest statistical significance are 
culled and interpreted. In this particular analysis, prior test 
score, student race, and the free/reduced lunch indicator were 
found to be "important" while the other variables were found to 
be "insignificant". As summarized in the district court 
judgement, "plaintiffs conclude from this particular analysis 
that the discriminatory treatment which produces classroom 
segregation in the lower grades also produces differences in BST 
[achievement] performance on the tenth grade level" (CV482-233 
U.S. District Court, Georgia p. 32). 

The usual interpretation of this kind of multiple regression 
analysis focuses on the notion that if a variable such as racial 
identification of the student appears important in the prediction 
of later achievement with prior achievement also used as a 
predictor, then the data indicate that different racial groups 
have different rates of academic progress. The question, Do 
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blacks and whites progress at different rates? Is a typical 
question regarding a correlate of change — that Is, Is racial 
Identity a correlate of change In achievement? Rogosa and Hlllett 
(1985) , In a technical critique and review of various methods for 
assessing correlates of change, find that these kinds of 
regression methods have very serious shortcomings and cannot be 
depended upon to identify correlates of change when such exists. 

Adjustment for diffe rences in initial status. A distinct, 
but conceptually similar, use cf regression methods arises in 
adjustments of later achievement levels or to gain in achievement 
for differences in initial status in variables such as initial 
achievement or demographic characteristics. The most common form 
of such adjustments comes in the construction of residualized 
change scores. Such ucol'^jB attempt to answer the question. How 
much would the students have changed if everyone had started out 
equal? or, equivalently, What would be differences in later 
status if everyone (i.e. different racial groups) had started out 
with equivalent initial status? A motivation for the use of such 
regression adjustments in both behavioral science research and in 
various court cases arises from concern that observed progress 
(e.g. the difference between later and prior achievement levels) 
is confounded or correlated with Initial status. For example, in 
the Tattnall County case (sections 2.2.3, and 3.1) it was shown 
that "students who made the largest gains in their CAT scores 
during their high school years were the students who scored at 
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the lowest levels on entry to high school.** (540 F«Supp. at 764). 
It Is often assumed that this kind of result is an artifact 
rather than an actual indication of differential student 
progress, and concern with such an artifact is used as 
justification for the construction of the residualized change 
score. Rogosa et al. (1982) demonstrate the highly unattractive 
properties of residualized change scores as a measure of 
progress, and Rogosa and Willett (1985) further demonstrate that 
correlations of residualized change scores with background 
variables such as racial identification or demographic 
characteristics serve very poorly to indicate variables that are 
associated with differential growth or progress. 

1.3.2.3. Comparison with a standard for student progress. 

Of most conse'^ence in assessing student progress is to 
interpret or judge the amount of student progress that the data 
indicate by comparison with a standard for some sort of expected 
progress. A question often asked is not **are students learning?*^ 
but, **are students learning enough?** The grade equivalent method 
of one unit per year of schooling is the most obvious example of 
such a standard and this appears in Scheelhaase v. Woodburv 
(section 2.1.1), NAACP v. Georgia (section 2.4.1.1), and San 
Francisco NAACP v. S.F.U.S.D. (section 2.3.2). Also, in NAACP v. 
Georgia f a standard of expected gain of one Normal Curve 
Equivalent per year is also presented by the defendant school 
districts. **In 1978, when the expected gain in number systems, 
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conputatlon, and application was 1 NCE, only three students from 
among over 200 tested failed to meet this expected gain** (CV482- 
233 page 23) . In Berry v. Benton Harbor comparison of achievement 
level with norms for the state of Michigan was used; such a 
comparison with a relevant group, when such exists, is another 
common form of assessing progress relative to a standard. Yet 
another implicit standard for progress is often attempted by use 
of test scores in the metric of percentile ranks. In particular, 
a notion of normal educational progress could be formulated by 
the standard of maintenance of percentile rank standing over 
successive years. Of course, this kind of standard raises the 
question of whether adequate student progress exists if a low 
scoring student at say, the 30 percentile rank manages to 
maintain that percentile rank over the course of school years. 
This is a controversial standard as many courts hope that 
educational progreuns, such as desegregation programs, will tend 
to decrease the gap between minority and majority students, not 
just allow the maintenance of such disparity. 

Regression ad justmen ts to growth rate. In the Tattnall 
County caue (section 2.2.3) analyses were presented that combined 
growth curve methods and regression adjustments for differences 
in initial status in a determination of whether student progress 
was "greater or less than expected." The analysis begins with 
the estimation of a straight-line growth curve for each 
individual using achievement scores at the four high school 
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grades. The estimation yields for each individual a slope (rate 
of progress) and an intercept (initial status at beginning of 
high school) . Then, from the (rate, intercept) pairs for each 
individual, a scatterplot is formed of rate (ordinate) versus 
intercept (abscissa) . Superimposed on this scatterplot is a 
straight-line fit of rate on intercept. Points (individuals) 
above the line are deemed to be making progress better than 
expected and those points below this regression line are deemed 
making progress worse than expected. Attention is also given to 
comparing this determination for individuals above and below the 
median on initial status; the regression lino and median split on 
initial status divide the scatterplot into four quadrants. 
Rogosa et al. (1982, p. 743) show the equivalence between 
residual change procedures and this type of regression adjustment 
to the estimated growth rate. 

student progress In ability grouping . In court cases 
revolving around the adequacy or non-discriminatory effects of 
ability grouping or tracking a common question is, *'Are students 
making adequate progress in each of the tracks?" One commonly 
used method is described by Findley and Bryan (1975) . Findley and 
Bryan present data from a southern school district on the amount 
of change students in varying ability-grouped levels attained 
over two years. The twenty-four comparisons within each ability 
level consist of eight subtest scores for each of three groups. 
"A group was said to have made better than expected progress on a 
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test if the percentile equivalent on the national noma of the 
mean at the higher grade was two or nore points higher than the 
percentile oqulvalent of the mean at the lover grade had been for 
that grade; likewise, a group was judged to have made less than 
expected progress if the percentile equivalent of the mean on the 
national norms at the higher grade was two or more points lower 
than the percentile equivalent of the mean at the lower grade had 
been for that grade** (p. 14). Table 3 reproduces their example 
which shows an apparent strong effect of tracking which is 
positive for high tracks and negative for low tracks. 



Insert Table 3 here 



The notion of evaluating the tracking system by measuring 
whether those in lower tacks maintained normal progress was also 
found in Mills and Bryan (1976) . **To determine whether ability 
grouping in a school system is beneficial or detrimental to 
students, HEW compares the achievement gain or loss for students 
in the low group with the achievement gain or loss for students 
in other sections— middle, high, etc.-- or the same grade at the 
same school. If, for example, the slow group at a particular 
school ranked at the thirtieth percentile when compared with the 
national norms at the ilrd grade and ranked at the fifteenth 
percentile when tested at the fifth grade, while the middle group 
at the same school ranked at the fiftieth percentile in both 
third and fifth grades, then the grouping has not resulted in 
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SECTION 1.3 

TABLE 3 

CHANGE IN PERCENTILE RANK FOR POUR LEVELS OF ABILITY GROUPING 
(adapted from Flndley and Bryan, 1975, p. 14) 

Better than Expected Less Than 

Level Expected Progress EXP^gte<l 

I (high) 18 0 6 

II 13 5 6 

III 5 7 12 

IV (low) 2 7 15 



ERIC ^ ^ 



equal educational benefit for the two groups" (Mills and Bryan, 
p. 43). 

1.3 .2. 4. Monitoring Student progress. 

Another fom of the assessment of student progress, denoted 
by the term **monitoring,'* uses much richer and much denser data 
collection stategies than is typically represented by yearly 
standardized achievement testing. One of the most prominent 
examples of monitoring of student progress arises in special 
education contexts reflecting the requirement for Individualized 
Educational Programs (IE?) mandated by Public Law 94-142. One 
quantitatively based system for monitoring student progress that 
is used especially in the Northwest is the Precision Teaching 
Model (described in White and Marring, 1980) . The Precision 
Teaching Model provides a way to account for individual student 
progress. A fundamental assumption underlyir^g the Precision 
Teaching Model is that learning is presumed to occur at an 
exponential rate. That means that achievement plotted against 
time will have a constant rate of increase if achievement is 
measured on a logarithmic scale rather than in the actual raw 
score or percent correct. The Precision Teaching Model provides 
an interesting departure from the constant rate of change models 
implicit in grade equivalent scores. 

This accounting method begins by identifying a specific 
objective (e.g., 90 percent correct on a spelling test of five- 
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letter coiamon words) , and a certain date for this objective to be 
reached. Initial status (e.g., score on the sane type of test) is 
plotted at the beginning date. The special graph paper is 
logarithmic on its ordinate, such that a straight-line on this 
paper represents a log function. The "minimum acceleration line" 
is drawn, connecting the objective with initial status (across 
time) . The student must progress at least at the same rate as the 
minimum acceleration line to reach the goal by a given date. 

The student's daily progress is plotted on the graph. If the 
observed points are lower than the expected rate for three 
consecutive days, a new intervention (i.e., different learning 
materials or methods) is implemented. If the student continues to 
fall below the expected line, the objective may be evaluated, 
restated, and even redefined. 

No attempt is made to compare progress across students. 
Rarely is the goal for a student to join regular classes, 
although, for borderline cases, often a student is in regular 
classes in the morning, for example, and special reading classes 
in the afternoon. Mobility across special education 
classifications (EMR, LD, ED, . . . ) or between special or regular 
classes is not common, and is rarely specified as an objective. 
1.3.2.5. Improvement in classification status. 

An important but qualitatively different kind of student 
progress arissui in court cases involving student classification: 
either into ability groups or classification of students into 
special education classes. The student progress being assessed is 
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not an Improvenent In achievement but whether or not changes In 
classification are non-discriminatory or appropriate ^ previous 
issues regarding ability grouping ask the question, "Given the 
ability grouping that exists, are students making adequate 
progress?*' Another aspect of that question is, **Are students 
reclassified into appropriate groups if they have a certain level 
of progress or educational improvement?** In the Dillon County 
case (section 2.4.1.3) the ability grouping of students was found 
to produce highly differential mobility patterns for black and 
white students. In cases such aa Hoffman v. New York (Section 
2.4.2.2) a key issue was whether classification into special 
education versus regular classes was sensitive to the progress of 
the student. 
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SECTION 2 

EXAMPLES OF EDUCATIONAL COURT CASES 
INVOLVING MEASUREMENT OF STUDEJIT PROGRESS 

The court cases involving measurement of student progress 
identified in our research are organized into four major areas 
which we term evaluations of teacher performance, rftcial 
discrimination . «»^*]l\lffiti9r« school desegregation Procrrftffis, and 
student classification programs . Major purposes of this research 
project are to catalog and better understand the types of data 
and summaries that have been used, and to suggest improvements in 
both design and analysis. Relevant cases were identified by 
searching descriptive phrases which provide relevant key numbers 
in the West's Digest and Federal Digest systems. Most casas were 
located under the descriptive phrases of: educational progress, 
student progress, student test scores, and student achievement. 

Evaluations of Teacher Performance: Court cases involving 
the evaluation of teacher performance are becoming increasingly 
visible dre to various state and federal interests in teacher 
accountability and competence, we would expect court cases 
regarding teacher performance evaluation to become increasingly 
common and more complex in the near future; the cases that we 
discuss, thus, may be regarded as prototypes of future 
controversies. A key legal standard in the evaluation of teacher 
performance appears to be whether a teacher's students are 
demonstrating "normal educational progress." Although that notion 
may appear unequivocal, in reality such a standard opens up a 
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plethora of types of data and data summaries of the student 
achievement scores that could be presented in legal proceedings. 
A recent Important example of legislative concern with the 
evaluation of teacher performance using student test data is 
California Senate Bill 813, the Educational Reform Act; a court 
case in which these issues are prominent that we consider in 
detail in this report Is scheelhaase v. Woodbury . 349 F.Supp. 988 
(1972) and 488 P. 2d 237 (1974) from the state of Iowa. 

Racial Discrimination ; Arguments about the educational 
progress of a particular minority group serve as important 
evidence in unequal educational opportunity cases. Here the 
plaintiffs argue that the school district is not providing equal 
opportunity to all its students. The main indications of unequal 
educational opportunity arise in either comparison of the 
minority group progress to that of a majority group, or by the 
comparison of the progress of the minority group with some 
expected or "normal" educational progress. Specific examples of 
discrimination cases involving evidence of student progress are 
Hobson V. Hansen 269 F.Supp. 401 (1967) and 327 F.Supp. 844 
(1971) in the District of Columbia; Serna v. Portalea in New 
Mexico, 351 F.Supp. 1279 (1972) and 499 F.2d 1147 (1974); Johnson 
V. Slkes and Anderson v. Banks (aka, "Tattnall County") 520 
F.Supp. 472 (1981), 540 F.Supp. 761 (1982), and 730 F.3d 644 
(1984) ; and Berry v. Benton Harbor 515 F.Supp. 344 (1981) . 

Evaluation of School Desegregation Pro grams; As part of 



desegregation decrees or other legal agreements, the monitoring 
of student progress after the implementation of desegregation 
plays an Important role. Concerns Include the effects of 
desegregation on the progress of minority students and also any 
Impact on the progress of the majority students. Two examples of 
monitoring of student progress subsequent to Implementation of a 
desegregation program are Berrv v. Benton Harbor (cited above) 
and San Francisco NAACP v. San Francisco Unified School District 
576 F.Supp. 34 (1983). Both Of these cases began as 
discrimination suits, which, after desegregation was ordered. 
Involved subsequent monitoring of student progress. 

Student Classification; A number of important court cases 
have involved assessments of student progress in ability 
grouping, tracking, and the assignment of students to special 
education classes. Several court cases involve assertions that a 
school district is practicing segregation under the guise of 
tracking, claims of classroom racial identif lability and unequal 
educational opportunity for students have been the impetus for 
court action. A major method of sorting students is to group by 
ability either within a classroom or into different classrooms. 
Classification issues also surface for epecial class assignment. 
Public Law 94-142, the Education for all Handicapped Children Act 
of 1975, assures all handicapped children the right to a free and 
appropriate education* Numerous Issues of educational progress 
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are derived from this act, including the annual evaluation of 
student progress, both to re-evaluate the classification 
decision, and to comply with the recpiirement for federal funding. 
Individualized educational programs (lEP) provide one interesting 
method of monitoring the educational progress of handicapped 
children. 
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2.1 EVALUATION OF TEACHER PERFORMANCE 
2.1.1 Scheelhaase v. Woodbury. 

The appropriateness of the evaluation of teacher perfomance 
was the key issue in Scheelhaase v. Woodbury . 349 F.Supp. 988 
(1972) and 488 F.2d 237 (1974), which was generated by the school 
district's failure to rehire a veteran teacher on the basis of 
her students' test scores. The teachers In the Woodbury School 
District are not tenured; rather, each year they are rehired at 
the district's discretion. Scheelhaase was one teacher who was 
not rehired; the school district claimed her performance was 
unacceptable. Scheelhaase contested the di8i:rict's decision and 
brought the matter to District Court. The school district 
examined achievement scores of students who had completed a year 
in Scheelhaase ' s 7th grade classroom and concluded that the low 
scores reflected poor teaching. "The specific reason given 
plaintiff for termination was her professional incompetence, a& 
indicated by the low scholastic accomplishment of her students on 
the Iowa Tests of Basic skills (ITBS) and Iowa Tests of 
Educational Development (ITED)" (349 F.Supp. at 989)^ 

In her complaint, Scheelhaase ' s expert witness, Norman 
Ashby, calculated the average gain per year of the entire 
school's students. These calculations were based on the average 
obtained grade equivalent scores on the subtests of the ITBS 
which was administered annually. The overall average student 
gains for each subtest were then treated as expected gains or 
standards. The gains of Scheelhaase ' s students were nearly the 



same as those averages, supporting the claim that her students 
made normal educational progress. (This analysis is considered in 
detail in Section 3.2) 

Neither the court nor the defendant school district 
questioned these calculations. "The district court, McManus, 
Chief Judge, held that termination of Iowa teacher's contract by 
nonrenewal thereof on grounds of professional Incompetence as 
indicated by low scholastic accomplishment of students on 
specified tests was arbitrary and capricious, since teacher's 
professional competency could not be determined solely on the 
basis of student's achievement on the tests, especially where the 
students maintained normal educational growth rates" (349 F.Supp. 
at 988-89) . Thus, the district court held for the plaintiff, 
Scheelhaase. 

The Appeals Court ruled for the school district. "The Court 
of Appeals, Talbot Smith, Senior District Judge, held that 
board's refusal to renew teacher's contract did not give rise to 
cause of action under Civil Rights Act" (488 F.2d at 238). 
"School board's decisions in exercise of its discretion are not 
vulnerable to correction by court merely if they are 'wrong, ' 
sustainable only if they are 'right.' ...Such matters as 
competence of teachers, and standard of its measurement are not, 
without more, matters of constitutional dimensions which permit 
court to overrule school board's exercise of its discretion" (488 
F.2d at 2jB) . 

This case provides one vivid example of the use of stud'^nt 
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gain or progress in evaluating teacher performance. In the 
Critique and Reanalysis Section (Section 3.2) we reexamine the 
evidence cited by both the District and Appeals Court. 
2.1.2. California Senate Bill 813 (1983). 

The Hughes-Hart Educational Reform Act of 1983 (SB 813) 
establishes a varlet:^ of programs designed to Improve the quality 
of elementary and secondary education In California. A key area 
of this reform Is teacher evaluation, and several sections of the 
bill address this Issue. The obvious goal of educational reform 
In California Is to Improve the quality of classroom Instruction. 
In this respect, then, all of the reform efforts eventually focus 
on the classroom. Teacher evaluation reform Is only one piece of 
this larger reform effort designed to Improve the quality of the 
student/ teacher Interaction. 

SB 813 requires local districts to make the following 
changes: 1. The local board must adopt standards of expected 
pupil achievement by which teachers will be evaluated. The use 
of publisher's norms established by standardized tests are 
explicitly prohibited for this use. 2. Local districts must now 
identify the range of instructional strategies and techniques 
which teachers are expected to use appropriately. 3. Local 
districts must now ensure that curricular objectives are in place 
for all subject areas. Teachers will now be held accountable for 
the teaching of these objectives. 

Issues related to expected pu pil achievement. Although SB 
813 mandates the districts to adopt standards of expected pupil 
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achievement, little explicit guidance is provided. Certainly, 
making explicit standards for expected pupil achievement is a 
difficult and complex issue. Although a great deal of research 
has been done regarding effective schools and effective 
instructional practices, no elegant technical solutions emerge to 
the problem of linking teaching behaviors to student achievement 
measures . 

In SB 813 the State has mandated that expected pupil 
achievement will be one of several criteria against which 
teachers in the state will be evaluated. Local districts are 
left to determine both the measures of student achievement and 
standards that will be used regarding this criteria. Indicators 
of achievement include tests, writing quality, or number of 
homework assignments completed. Standards, on the other hand, 
represent the level of a chosen indicator that is deemed 
acceptable. For example, one standard might be that 80% of a 
teacher's students should achieve a score of 70 or above on a 
district exam. 

The use of student test scores alone as a standard of pupil 
achievement assumes a degree of comprehensiveness for such tests 
that they often do not possess. Exclusive reliance on objective 
paper and pencil tests may leave many important educational goals 
unexamined. An alternative might be the adoption of standards on 
a variety of measures of student achievement (homework, projects, 
essays, standardized tests, teacher constucted tests, and the 
quality of classroom discussions). (See Haertel, 1986) 
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standardized tests matched to a district's curriculum do not 
exist In many subject areas. It may not be possible or wise for 
each district to spend time and money in the development of 
additional student tests to use in the evaluation of teachers. 
How to use existing measures in the construction of standards for 
expected student achievement is a complicated but important 
problem that SB 813 presents for local districts. 

The educational reform movement in California and SB813 in 
particular, have attracted nationwide attention. The central 
role of expected studont performance in the evaluation of 
teachers is likely to be a central issue of debate in any 
deliberation involving tveacher dismissal or other action that is 
contested. The reason for including this legislative act in the 
review of court cases is its likely role in future court cases, 
along with its emphasis on the assessment of student progress. 
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2.2 EXAMPLES OF RACIAL DISCRIMINATION CASES 
2.2.1. Hobson V. Hansen. 

In Hobson v. Hansen . 269 F.Supp. 401 (1967) and 327 F.Supp. 
844 (1971) , the District of Columbia School System was found to 
"unconstitutionally deprive the district's Negro and poor public 
school children of their right to equal educational opportunity 
with the district's white and more affluent public school 
children" (269 F.Supp. at 406). The court ordered a desegregation 
remedy which consisted of a voluntary busing program. The 
background for the 1971 case was described by the court as: "In 
an effort to diminish the discrimination against children In the 
overcrowded schools east of the Park, this court In Its 1967 
decision ordered the Board to bus, on a voluntary basis, the 
primarily black and poor children from the overcrowded schools to 
the underpopulated, predominantly white nonpoor schools west of 
the Park. Several thousand children have been participating In 
the program during the past three years. Achievement test results 
from these children taken before they left the sending school 
compared with their annual test results at the receiving schools 
would have provided an Indication, at least, of any discrepancy 
In the quality of the education available at the schools of one 
side of the Park vls-a-vls the other side" (327 F.Supp. at 858, 
footnote 26) . 

The meet Interesting and Important aspect of this case from 
our standpoint Is the court specification of what would 
constitute useful and compelling evidence of the adequacy of the 

2-10 



80 



district* 8 desegregation actions to that date. **In an effort to 
suggest to defendants the kind of evidence they should be 
presenting if they were to prevail, the court ordered 
on January 28, 1971 that 'defendants file in the record not later 
than February 15, 1971 such statistics and studies as will show 
the effect of the voluntary busing program on the achievement 
test scores of the children participating. These statistics 
should be on a school by school basis so that the improvement, if 
any, of the children at each receiving school may be discerned"* 
(327 F.Supp. at 858). 

The court's idea in issuing this order was that a study of 
the Improvement or lack of improvement in achievement test 
performance by students in the voluntary busing program, by 
providing a control for the factor of socioeconomic background, 
would be probative of whether schools west of the Park provide a 
better education than do schools in the rest ot the city. On 
February 16, 1971, however, defendants moved the court to rescind 
this order on the ground that it imposes an unduly burdensome 
task or the defendants. The gist of the memorandum in support of 
the defendant *s motion was that no systematic records of test 
results had been kept, and that those bused children who had been 
tested had been given different brands of tests for which 
conversion scales are unavailable — thus rendering meaningful 
comparisons impossible. *'While the court does not charge the 
defendants with a lack of candor, it does seem incredible that a 
school system under injunction to provide equal educational 
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opportunity to all Its Btuder.ts would not have shown ikore 
Interest In studying the effect upon Individual student 
achievement of a voluntary busing program which permits students 
to transfer from allegedly inferior to allegedly superior 
schools. That defendants have failed to keep any systematic 
records of the achievement test results of these bused students 
raises questions about their effectiveness as administrators, if 
not about their good faith as parties to this case.** (327 F.Supp. 
at 858-59) . 

What makes this case unusual is the clear emphasis on the 
effect that the busing program had on student progress as opposed 
to the less adequate but far more common reliance on a static 
assessment«-how well do fourth grade students do when compared to 
national norms? — which does not serve nearly as well to reflect 
the effects of the desegregation program. To be specific on the 
nature of the usual static comparisons, consider this example of 
the plaintiff's evidence that unequal educational opportunity 
still existed in the district. "Plaintiff's prima facie case of 
discrimination in the provision of educational opportunity, based 
upon the pattern of unequal expenditures which favors the schools 
west of the Park, is strongly buttressed by further evidence in 
the record concerning the results of city-wide sixth grade 
reading achievement tests. The record shows that the west of the 
Park elementary schools produced an averagf^ reading achievement 
test score that was significantly higher— ^indeed 2.4 grades 
higher — than the average for the rest of ^:he city. Obviously, 
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these resultM tend to corroborate the presumption created by the 
pattern of expenditures that the city provide a better 
educational opportunity to Its richer, white students. 
Defendants' dubious argument that the smaller classes and higher 
proportion of experienced teachers In the schools west of the 
Park do not give students there a better chance for a good 
education than can be had elsewhere In the city Is still less 
convincing In light of this testing evidence.** (327 F.Supp. at 
858) . Although this kind of static comparison is useful in 
identifying potential denial of equal educational opportunity, it 
is not adequate to assess the efficacy of an attempt to 
remediate u Inequities in test scores can be due to many other 
factors totally separate from the school district *8 provision of 
equal educational opportunity. 

2.2.2 Serna v. Porta3.e8 . 

This case, 35X F.Supp. 1279 (1972) and 499 F.2d 1147 
(1974), Involved the Lindsey Elementary School In New Mexico and 
the education of Spanish-surnamed students. Plaintiff claimed 
that the school district was not supplying equal educational 
opportunity to the Hispanic students. The evidence presented 
consisted of both IQ and achievement data of the Lir^dsey School 
compared with the other three elementary schools (predominantly 
white) in the Portales School District. Lindsey School was the 
only school in the district with bilingual, bicultural programs. 
The evidence presented by the plaintiffs consisted primarily of 
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the usual static comparisons of a grade level achievement of the 
minority school versus the other schools. The court found that 
**the evidence presented, indicates that the achievement of 
children at Lindsey is consistently lover than that of the 
children attending the other three elementary schools. I.Q. 
tests administered to fifth grade students in the four Portales 
municipal elementary schools reveal that the children at Lindsey 
scored approximately 13% lower than children at James and 
approximately 8% lower than children at steiner .... What 
becomes apparent from an examination of these scores is that the 
performance of the children at every level at Lindsey School is 
not what it should be when compared to the performance of 
students at the other schools'* (351 F.Supp. at 1281-82) . 

In the school district's appeal to the U.S. District Court, 
additional issues of cohort comparisons arise which suggest much 
better analyses regarding educational opportunity. "Undisputed 
evidence shows that Spanish-surnamed students do not reach the 
achievement levels attained by their Anglo counterparts. For 
example, achievement tests, which ars giver; totally in the 
English language, disclose that students at Lindsey are almost a 
full grade behind children attending other schools in reading, 
language mechanics and language expressions. Intelligence 
quotient tests show that Lindsey students fall further behind as 
they move from first to the fifth grade. As the disparity in 
achievement levels increases between the Spanish-surnamed and 
Anglo students, so does the disparity in attendance and school 

2-14 



dropout rates" (499 F.2<1 at 1X49-50) » Tha important point the 
District Court raises here is the notion that the static 
comparison at a single grade level does not tell the full or 
probably the important story; it is only by tracing the progress 
of cohorts of majority and minority students that one can find 
compelling evidence to adjudicate questions of "equal" 
performance or equal educational opportunity. 

2.2.3 Tattnall County. 

Tattnall County is how we refer to the series of cases 
(Anderson v. Banks and Johnson v. fiikes. 520 F.Supp. 472 (1981); 
540 F.Supp. 761 (1982); 730 F.2d 644 (1984)) in which important 
issues about the use of test scores and student improvement in 
achievement entered into the Court's decisions. Here we give the 
background of the cases pertaining to discrimination issues. We 
will return to this case when exeunining classification issues 
(Section ;.4.1.2), and again in the Critique and Reanalysis 
section (Section 3.1), where we examine the statistical analyses 
of the test data presented to the court and describe our re- 
analyses and recommendations for data collection and analysis. 

The Claim. The Tattnall County School District implemented 
the policy that high school students must be at at least the 
ninth grade level (6E-9.0) on the California Achievement Test 
(CAT) in reading and in mathematics in order to obtain a high 
school diploma. This policy was first instituted with the 
graduating class of 1978. The test has had vast differential 
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racial inpact: most of the students who did not pass were black. 

Desegregation and Implenentatlon of the unitary school 
system In Tattnall County began In the 1970-71 school year. 
Before this time, the blacks In the segregated dual school system 
had substandard educational cpportunltliis. The plaintiffs In this 
case felt that the diploma policy was racially discriminatory, 
given these past educational Inequities. 

Another relevant Issue, following the case of Debra P. v. 
Turlington (474 F.Supp. 244, 644 F.2d 397), is when »the award of 
diploma depends on outcome of a test, burden is on school 
authorities to show that test covered only material actually 
taught" (644 F.2d at 475). 

History . The Tattnall County schools were totally 
segregated until 1965. A voluntary integration system, 
encouraging blacks to attend the better white high schools vas 
initiated, though it never got off the ground. 1970-71 was the 
first year the blacks and whites attended the same school. 
Despite the unitary system, no integration attempts were made: 
the buses carrying students to school were segregated, two buses 
for the blacks and two buses for the whites. There were no 
programs to help the students adjust. At the same time tha 
unitary system was initiated, a tracking system was set up, which 
the school district claimed was grouping by achievement. Yet the 
tracking often resulted in racially identifiable classrooms with 
the higher tracks predominantly white, and the lower tracks 
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predominantly black. Racially Ident if labia classrooms continued 
until the 1979-1980 school year, when the tracking system was 
abandonod as a result of an OCR Investigation. (More on 
Tattnall's tracking system under classification Issues.) 
The test 

The CAT was chosen by the school district without doing a 
local validation. The diploma requirement and establishment of 
remedial classes were passed by the school board in 1976 ^ and 
were Imposed two years later, beginning with the class of 1978. 
That class was administered the 197 > CAT. Subsecpient classes had 
the 1977 C/-T formd C and D. CTB/McGraw"Hill developed the CAT. 
Grade-equivalency scores were derived from the norming sample, 
steady progress through the school year was assumed in 
interpolating the grade equivalent scores for each month. 

The plaintiff's expert witness. Dr. Shapiro, performed an 
item analysis on the CAT for Tattnall County using polnt- 
blserial correlations. He found that many of the items were 
biased in this population. In some instances the reliability of 
the CAT was quite unstable. It was assumed that progress from 
year to year should be steady, yet several Instances were 
discussed where vast Irregular fluctuations occurred from one 
year to the next. Yet overall, (steady) progress was made. The 
plaintiffs contend that it was the remediation program, and not 
the implementation of the exit exam, that was responsible for the 
improved scores. 
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The Court's daclalon . In order for the diploma sanction to 
violate the equal protection clause, It nust be shown (1) that 
discriminatory racial impact occurred — which In this case is 
undisputed, and (2) that the policy was adopted with 
discriminatory purpose. The court did not find evidence of 
discriminatory purpose in Tattnall County's attempt to improve 
the value of a high school diploma. But the court also took into 
account the county's dual system history. "If present racially 
neutral actions serve to perpetuate the past intentional 
discrimination, there is no requirement that intent [(2) above] 
bo proved again .... Thus, insofar as the poor performance of 
the black children is attributable to their participation in the 
dual system, the diploma sanction must fall. It cannot be 
constitutionally imposed on those who attended classes in the 
dual system" (520 F.Supp. at 500). "Clearly, no diploma sanction 
may be imposed until June, 1983, when those graduating will never 
have been exposed to the dual system" (520 F.Supp. at 503). 

Hill the diploma sanction be allowed after June of 1983? 
"Since the CAT is being used to measure what It was designed to 
measure, i.e., relative achievement levels in mathematics and 
reading, the court is of the opinion that local revalidation was 
not necessary" (520 F.Supp. at 507). Despite the examples cf 
anomalous student changes (which will be discussed below) as an 
indication of the test's unreliability, in addition to the 
earlier discussion of the item bias study indicating an Invalid 
test in Tattnall County, the "court cannot conclude that this 
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well-respected test Instinment is unsuitable for use In Tattnall 
County** (520 F.Supp. at 508) . 

But the curricula natch Is a different story. Drawing on 
Debra P., **the court can only conclude that where the award of a 
dlpl&jaa depends on the outcone of a test, the burden Is on the 
school authorities to show that the test covered only material 
actually taught . . . the court must conclude that the school 
authorities have not met the test of Debra P. . . . Because It 
has not been demonstrated that the Items on the CAT were actually 
taught In the Tattnall County schools, the use of the CAT as an 
exit exam must fall on substantial due process grounds" (520 
F.Supp. at 509). As Debra P. *s decision occurred simultaneously, 
the school district was Initially unaware of this burden; thus, 
a new trial was granted for the sole purpose of displaying the 
curricular validity. 
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2.3 Examples of Evaluations of Mandated Desegregation Programs 
Findings of discrimination are often followed by mandated 
remedies, either within the discrimination case or In a 
siibseguent court action. Student progress, especially of the 
affected minority students, Is a central concern In the success 
of the desegregation program In redressing previous 
discrimination . 

2.3.1 Berry v. Benton Harbor. 

Previous court opinions (442 F.Supp. 1280, 467 F.Supp. 630, 
and 494 F.Supp. 118) In 1977 and 1978 determined that the school 
district of Benton Harbor, Michigan was liable for 
''unconstitutional segregative conduct** throughout the school 
district through actions to ** Isolate black students In certain 
schools and within certain classrooms while preserving the 
predominantly white character of ether schoc'ls and classrooms** 
(515 F.Supp. at 348) . 

The purpose of the present case Is **establlshment of a 
constitutionally acceptable, fair, understandable and workable 
remedy** (515 F.Supp. at 348) . In framing the remedy for previous 
discrimination, **the court concluded that system-wide segregation 
such as the Benton Harbor District has created and the State 
Board of Education condoned and perpetrated by its inaction, 
result in measurably reduced achievements . . The problem of 
most concern to the court in addition to the racial separation 
and isolation of large numbers of children in these three school 
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districts is the chronic low achievenent levels of students in 
the Benton Harbor system. Dr* Stolee, the court's appointed 
expert, has reconmended that its plan include a program to 
improve the quality of instruction provided to the students in 
the Benton Harbor School District and to raise the level of 
student achievement until achievement within the district reaches 
the average attained by Michigan students in the Michigan 
educational achievement program. Establishment of an achievement 
component in the Benton Harbor schools is crucial to any complete 
and effective remediation of harm that has resulted from the 
defendant's unconstitutional segregative conduct** (515 F.Supp. at 
369) . 

A particularly interesting aspect of the court's remedy is 
the operationalization of the achievement goa"* . In describing the 
social skills and achievement component of the court's plan, the 
explicit statement is that "the goal . . . will be to raise the 
level jf student achievement, until achievement within the 
district reaches the average attained by Michigan students in 
statewide achievement tests. Once that goal has been achieved, 
t>:8 program will be phased out over two additional school years" 
(515 F.Supp. at 370). Thus, this case provides an example of the 
monitoring of student progress as compared to a common norm or 
standard. 

2.3.2 gan Francisco NAACP v. San Francisco Unified School 
District. 
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Itils ca8« b«gan as Johnaon at al. v. Sa n Franelseo Unified 
School Dl«trlct at al. (339 P.Supp. 1315 (1971); 500 F.2d 349 
(1974)), a school dasegragatlon case In which it was found that 
"for a long period of time there has been and there now is ^ 
jure segregation in the San Francisco public elementary schools. 
This segregation must be eradicated forthwith" (339 F.Supp. at 
1323). The district was ordered "to carry out, effective at the 
start of the next term of schools on September 8, 1971, 
desegregation of the student bodies of each and all of the 
8c^ools as provided for by the Horseshoe Plan or by the Freedom 
Plan ... To make bona fide, continuing and reasonable efforts 
during the nesct five years, to eliminate segregation in each and 
every school" (339 F.Supp. at 1324). Also the district was 
enjoined from a number of practices including "authorizing, 
permitting, or using tracking systems or other educational 
techniques or innovations without effective provisions to avoid 
segregation" (339 F.Supp. at 1325) . 

In 1983, the Northern California U.S. District Court 
considered a second case against the San Francisco Unified School 
District brought by the San Francisco NAACP (576 F.Supp. 34), in 
which a detailed consent decree was agreed to. Of particular 
Interest Is the portion of the consent decree that addresses 
academic excellence: "The decree directs the districts to 
monitor test scores and academic results in order to evaluate the 
continued effort to achieve academic excellence throughout the 
system" (576 F.Supp. at 42). 
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This casa Is imlqua In the •mphasls given to the nonltoring 
of student progress as a means to evaluate the efficacy of the 
desegregation plan. The 1983 court order specifies a consent 
decree which states in Paragraph 39 that **the S.F.U.S.D. shall 
evaluate student academic progress for the purpose of determining 
the curricula and programs most responsible for any improved test 
scores and learning in the District and the extent to which these 
curricula and programs are available to students of all 
racial/ethnic groups. The S.F.U.S.D. shall adopt any additional 
curricula and programs necessary to promote equal educational 
opportunity** (Consent Decree, p. 19) . Paragraph 40 the Consent 
Decree requires an annual report which '*shall include a section 
on S.F.U.S.D. * 8 progress toward the goal of academic excellence, 
setting forth test scores and other evaluative data for each 
building and for the District as a whole** (Consent Decree, p. 
19) . This case — the consent decr&e and the subse^ent yearly 
reports— provides a special resource for the study of data on 
student progress and its use in judicial decisions. 

The 1983-84 Annual Report pursuant to the Consent Decree by 
the San Francisco Unified School District (No. c-78 1445 WHO) 
reports the limited progress that the s.-hool district had made to 
that point. The 1983-84 Annual Report states that *'The San 
Francii3co Unified Sch^vol District was directed to (1) determine 
curricula and programs most responsible for academic development" 
and '*(4) provide test scores and other evaluative data for each 
school and the District as a whole'* (C-78 1445 WHO, p. 15) . At 
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the time of this report the district had aade United progress. 
**The District had not fully net this requirenent. The District 
did present infomation about curriculun prograns contributing to 
acadenic developnent, the selection and/or adoption of such 
prograns and an indication that no racial or ethnic restraints 
denied access to these prograns" (C-78 1445 WHO, p. 15). 

The nost recent report, the 1985-86 Annual Report by the San 
Francisco Unified School District to the U. S. District Court, 
Northern District of California (No. c-78 1445 WHO) stated the 
first 1985-86 responsibility of the school district to satisfy 
Paragraphs 39-41 of the Consent Decree as "detemine the 
cm^ricuia and prograns nost responsible for inproved test scores 
and learning by evaluating student academic progress'* (C-78 1445 
WHO, p.l; enphasis added). The court found the school district 
to have fully satisfied this and other decree responsibilities. 
The court stated "during the 1985-86 school year, the San 
Francisco Unified School District using 1984-85 data has 
evaluated student acadenic progress for the purpose of 
detemlning the curricula and prograns nost responsible for any 
inproved test scores and learning in the district and the extent 
to which these curricula and prograns are available to students 
of all racial/ethnic groups." (1985-86 Annual Report, Vol, 4, p. 

1). 

The enpirical evidence assembled by the San Francisco 
district sought to identify schools which have: "1. Consistently 
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high relative and absolute student test perfomanoe and, 2. 
Schools where students rates of growth are faster than the 
national norm** (1985-86 Annual Report, Vol. 4, p. 2> . The 
Calif oimla Assessnent Program scores were used for point 1. In 
particular, the district presented for each grade level tested by 
CAP (grades 3,6,8,12) the number of schools whose scores exceed 
the top of their comparison score band. The comparison score band 
is a statistical range constructed so as to Include the middle 
50% of the distribution of schools with demographic 
characteristics that are similar. Thus exceeding the comparison 
score baT>d places a school's score in the top 25% of other 
statistically similar, in demographic terms, schools. 

To identify schools with student growth rates faster than 
the national norm, the Comprehensive Tests of Basic Skills (CTBS) 
scores of fall of '84 and fall of *85 vrere used. The district 
calculated the ** amount of growth experienced by students at each 
grade level at each school** by calculating the difference between 
grade equivalents scores of fall *85 and fall *84 testing, a 
simple difference score in the grade equivalent metric. Expected 
growth was defined by **natlonal norm of ten months growth between 
annual testing.** (1985-86 Annual Report, Vol. 4, p. 5). The 
district stated **Our strategy here was to identify grade cohorts 
within schools where the growth rate was greater than ten months. 
These were identified as grade levels where the programming was 

« 

exemplary** (1985-86 Annual Report, Vol< 4, p. 5). The district 
found that in the *84-*85 data, 222 of the possible 342 
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eldraentary grada levelis surpassed this national average. 
Purthermore the district reported, that the same analysis carried 
out on *83-*84 data revealed 195 of the possible 350 elenentary 
grade levels surpassed the national average (i.e. the expected 
growth of 5/6 of a grade equivalent unit.) Furthemore, the 
district carried out the analysis of gains in grade equivalent 
units focusing on the original consent decree schools (see 1985- 
86 Annual Report, Vol. 4, p. 6). Analyses were also conducted on 
scores from those students who remained in their school for the 
full year (fron fall *84 to fall '85). Achievement gains in 
terms of difference scores of grade equivalent scores were 
computed and found often to exceed the average rate of 
improvement of the district overall, even though th^ mean pretest 
and mean postteet scores for these schools were lower than that 
of the district average. These results were interpreted in a 
positive light as "any achievement rate greater than that of the 
district means that students were catching up to the district 
average** (1985-86 Annual Report, Vol. 4, p.7) . 

Section 2.3.3. San Diego City School district. 

On December 2, 1980, the Superior Court of California, 
County of San Diego, issued to the Board of Education of the San 
Diego Unified School District a court order in the case of Karl 
Carlln et al. v. the Board of Education of the San Dieao Unified 
School District (No. 303800, Order Re Integration Plan 1980-81) . 
The court order stated in part the following: "It is ordered 
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adjudged and decraad that$ the Board of Education of the San 
Diego Unified School District will: 1. implenent a course or 
coursss of study in all minority isolated schools which will 
result, by the dates indicated in the table below, in 50% of the 
students in the isolated schools achieving at or above the 
national norm on the Comprehensive Tests of Basic Skills (CTBS) 
in reading, mathematics and language** (Re Integration Plan, p.l). 
The court *s table started with kindergarten grades achic^ving this 
criterion by 1982, first and second grades by 1983, third through 
sixth grades by 1984, and seventh through eleventh by 1985. 

In the fall of 1962 the court ordered additional analyses to 
provide a comparison of achievem<9nt levels in the minority 
isolated schools with that in the other schools in the district 
at all grades tested district-wide. In 1982 the court also 
requested the test data bo provided by ethnic subgroups. On Hay 
twenty-first, 1985 the court issued its final order which 
incorporated all pertinent past orders including those relating 
to achievement scores. It ordered that the district make an 
annual report to the court by September first of each year in a 
format that would permit meaningful comparisons from year to 
year. 

The explicit quantitative criteria set forth by the court 
make this case interesting, if not unique, as a case study in the 
use of test scores in desegregation casen. The district *s 
reports give the percent of pupils scoring at or above the 
publisher's median for non-minority isolated schools and minority 
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isolated schools for each of the najor ethnic groups (Hispanic, 
white, black, Asian, Alaskan/Indian) for different years of 
testing. The most recent data are presented in Report number 425 
of the San Diego City Schools Planning, Research, and Evaluation 
Division, **Testing Results fron Minority Isolated Schools, Spring 
1986** dated September 2, 1986. Figure 1 reproduces typical pages 
fron Report 425. The longitudinal aspects of these data di^^plays 
are quite interesting &b the progress depicted is in terms of 
successive cohorts rather than of individual students improving 
over time. The trajectories are given to show how successive 
cohorts progress la constrained in order to meet the court 
mandated criteria of more than 50% of the students exceeding the 
publisher's norm. Although the court's criterion in any given 
year is a cross sectional one for each cohort, the role of 
individual student progress ma still be important. These data 
show unusual patterns of moveiaent towards the court's criteria in 
the minority isolated schools, which may be due to cohort 
differences in improvement or to uneven progress by .Individual 
minority students as they move from one grade to the next. Such 
patterns of individual learning may severely confound the cohort 
by cohort comparisons featured in the district report. 



Insert Figure 1 here 
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CRAPHIC OISPLAY OF STUDENT PROGRESS 
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2.4 Examples of Student Classification Cases 

Our heading "student classification** enconpasses two 
distinct kinds of topics of litigation: Ability grouping of 
students within classrooms and within schools, and assignment to 
special education classrooms* 
2.4.1 Ability Grouping or Tracking. 

Key issues relating to student progress in schools involve 
the efficacy or lack thereof of tracking. In some cases, the 
claims of racial identif lability and unequal educational 
opportunity for students in lower tracks have been reasons for 
court action. Other litigation involving ability grouping is 
separate from the termination of racial discrimination. Examples 
of court cases which involve assertions that a school district 
prac'^ices follow the general discussion of ability grouping. 

One method of tracking is to group students by eUdility. 
"Ability grouping is the practice of organizing classroom groups 
in a graded school to put together children of a given age and 
grade who have most nearly the same standing or measures or 
judgments of learning achievement or capability** (Findley and 
Bryan, 1975, p. 9). 

Findley and Bryan (1975) posit 35 conclusions regarding the 
practice of ability grouping. Their discussion draws from the 
results of the U.S. Office of Education Task Force, In which 
Findley was the principal investigator. The conclusions regarding 
monitoring student progress follow. 

**Huch reliance is placed on early determinations of 
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capability. Individual children are often not reevaluated for 
different group placement .... Initial differences are thus 
allowed to cumulate no that low groups fall progressively further 

behind** (p. 21). 

Flndley and Bryan urge that "provision should be made for 
frequent review of each individual's grouping status as part of 
the instructional program. The evidence that ability grouping 
results in practically permanent assignment of children to low or 
high groups . . . makes a regular program for reviewing group 
placement absolutely essential" (p. 27) . Inherent in the plea for 
mobility between tracks is some method for measuring an 
individual's progress. 

Another conclusion reads that the practice of ability 
grouping produces "almost uniformly unfavorable evidence for 
promoting scholastic achievement in average — or low — achieving 
groups" (p. 13). A description of Flndley 's method for the 
analysis of scholastic achievement was given in Section 1.3. An 
example in that discussiori displayed mostly less than expected 
progress for the low ability levels. Not all the studies 
assessing the change of lower track students show such losses. In 
view of the conflicting results, researchers "agree that the 
evidence cannot be used to support the assumption that ability 
grouping aids achievement. Furthermore, these reviewers conclude 
that where research studies cite achievement gains in favor of 
homogeneous [ability] grouping, results are often explainable on 
the basis of differing teaching methods and materials, 
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modification of educational objectives, and curriculum 
reorganization" (Wilson and Schmits, 1978, p. 536). 

Oakes (1985) surveyed secondary teachers regarding their 
grouping practices. Shes concludes that ability grouping 
persists not only out of tradition, but because teachers believe 
that students profit most if they are with those "who learn at 
the same rate" (p. 6). Oakes* sentiments seem to be 
representative of current educational researchers. "As we know 
from the research . . . tracking . . . does not appear to be 
related to either increasing academic achievement or promoting 
positive attitudes and behaviors" (Oakes, p. 191) . 

All the researchers reviewed believe that the best grouping 
practice consists of small groups of individuals with 
heterogeneous abilities, sometimes referred to as the "Baltimore 
plan." If ability grouping must persist, then mobility between 
tracks, and thus careful monitoring of progress, must prevail. 

2.4.1.1 NAACP V. St^tte of Georgia. 

This case, NAACP v. S tate of Georgia (CV482-233 U.S. 
District Court, Georgia) involves the contention that ability 
grouping is an attempt to disguise intentional racial 
discrimination. In the ruling that certified class action status 
for the plaintiffs' suit, (99 F.R.D. 16 (1983)), the complaint 
was summarized as: "The plaintiffs seek declaratory and 
injunctive relief against the defendants in order to end alleged 
intentional racial discrimination in the public schools in the 
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state of Georgia. Plaintiffs contend that black school children 
in Georgia, including the individual plaintiffs and the members 
and children of the state conference's branches in Liberty County 
NAACP havQ been assigned to classrooms within biracial schools on 
the basis of their race. The plaintiffs allege that the 
assignment of these children to disproportionately black classes 
results from the operation of intentional racial discrimination 
rather thar^, racdally neutral criterion such as ability or 
achievement grouping. Plaintiffs claim that as the result of the 
disproportionate classroom configurations they and the class they 
seek to represent have suffered severe educational deficits which 
must be remedied** (99 F.R.D. at 19). 

The case also contains a special education component which 
is summarized as: i^Plaintiffs also seek relief from allegedly 
racially discriminatory administration of the special education 
program in the state of Georgia. It ij alleged that black school 
children are erroneously classified as being educable mentally 
retarded as a means of removing them from normal classrooms or 
excluding them from programs for specific learning disabled 
children. Plaintiffs further contend that this misclassification 
of black children is racially motivated and has a devastating 
effect on these children's lives because of the stigma associated 
with being identified as mentally retarded and the diminution of 
their educational opportunities. Finally, plaintiffs claim that 
the misclassification of non-handicapped or specific learning 
disabled children as educable mentally retarded also excludes 
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then from appropriate academic placements because of a real or 
attributed handicap** (99 F.R.D. at 19). 

In 99 F.R.D. 16, the district court certified class status 
for the plaintiffs and the case was then tried in the U.S. 
District Court for the Southern District of Georgia, Savannah 
Division. The two basic claims of the case are **(1) that black 
children have been tracked into racially malapportloned 
classrooms in otherwise integrated schools; and (2) that black 
students have been mlsclassifled as being Educable Mentally 
Retarded (EMR) because of their race** (99 F.R.D. at 19-20). 

The court ruled fur the defendant school districts on both 
claims. The court found no Intentional discrimination on the 
basis of race or handicap. Some of the evidence supporting the 
defendant school district *s case (to the dismay of the 
plaintiffs) included student gains. '*The local defendants 
demonstrated that significant academic gains have been made by 
students educated within the structured learning environments. 
Effectiveness of the ability grouping models for children 
Instructed in the low achievement groups was particularly 
emphasized** (Civil Case Judgment Marshall v. State of Georgia 
CV482-233, p. 22). 

It is of particular Interest in this research to examine the 

defendant *s s'/ldence of student progress. Some of this evidence, 

excerpted from pages 22-24 of CV482-233, follows. 

*'The students participating in Chapter 1 programs 
in the Vldalia City schools have consistently exceeded 
the gains expected pursuant to the approved Chapter 1 
program. In 1978, when the expected gain in number 
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systems, conputatlon, and application was 1 Normativs 
Curve Equivalent (NCE) , only three students from among 
over 200 tested, failed to meet this expected gain. The 
remaining students experienced NCE gains ranging from 
1.8 to 6.1< Likewise, in 1980, the actual NCE gain 
experienced ranged from 2.4 to 19.9 NCE*s. Similar 
progress was shown for 1982. Vidalia also illustrated 
its improvement in performance on the 6CRT in math and 
reading. An analysis of test results for the Spring of 
1981 and 1983 demonstrates an average increase in 
objectives achieved out of a possible total of 20 of 
4.04 by Chapter 1 students while non-Chapter 1 students 
experienced an increase of average objectives of only 
1.27. 

"Lee County showed similar gains for its Chapter 1 
students. In the 1980-81 school year, the average 
improvement shown by these students, in grade 
equivalence, was 1.45 grades in reading and 1.90 grades 
in math. In the next term, the average improvement was 
.88 grade equivalence in reading and 1.38 in math. 
Effectiveness of Lee County's educational methods for 
all students was demonstrated by the students* 
achip\'ement on the 6CRT. For example, in 1983 the 
second graders exceeded the Georgia state average in 16 
of 20 math objectives measured, equalled the state 
average in 3 of those objectives, and fell below in 
only 1. In the same year, the sesond graders* reading 
scores exceeded the state average in 24 of 25 
objectives measured and equalled the state average in 
the 25th. Similar results were experienced by Lee 
County *s third and fourth graders. 

*<Crisp Coiinty*s students have also exhibited 
academic progress since the introduction of achievement 
grouping. This is seen in 6CRT results between 1976 and 
1980, during which time the percentage of fourth 
graders who {satisfied 6CRT reading and math objectives 
rose from 59.2% to 80% and from 59.1% to 77.5%, 
respectively. For the same period, the percentage of 
eighth graders who satisfied the 6CRT objectives 
increased 26.2% for reading and 19.4% for math. 

''Criterion Reference Test tables for Evans County 
reveal that its students have also made significant 
academic gains. Over I'ae past five years, Evans County 
children have outscored their counterparts across the 
First District in 82.5% to 97.5% of the test objectives 
and outperformed the state as a whole in 60.0% to 92.5% 
of the 6CRT objectives.** 

The plaintiffs presented a number of statistical analyses 
using student test scores which they argued showed harmful 
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fiducatlonftl Impact of tho ability grouping. This evidence was not 
found to be persuasive hi the judgment. (See pages 30-38.) 
2.4.1.2 Grouping Practlce<» In Tattnall County Case. 

Analvsls of Tracking System . The court found the 
disproportionate; number of blacks in the lower tracks difficult 
to justify. Based on an available sample of IQ scores, the court 
concluded that "the black and white children who were later 
subjected to the CAT diploma requirement began their academic 
careers with generally equal abilities'* (520 F.Supp. at 481). The 
school district claimed that track placement was based on 
achievement, not ability. The following ii?ere used as criteria for 
track assignment: Scott-Foresman Initial Placement Score, teacher 
judgment, and CAT scores. The children are grouped together for 
all academic subjects. The lower groups were taught essentially 
the same material as were the higher groups: No remediation was 
offered. 

Collins High was the one Tattnall County school that did not 
practice the tracking system. These students served as 
comparisons for the performance of those students who were 
tracked. Two standardized tests served as the beisis for 
comparison: The Georgia Criterion Referenced Test (6CRT) and the 
California Achievement Test (CAT) . For eighth graders in 1978 and 
1979, the students from the non-tracked school were dramatically 
ahead on both the 6CRT and CAT* The principal of Tattnall 
Elementary School submitted evidence that compared •♦pass rates" 
(i.e., proportion of students exceeding 9.0 on thct CAT) of high 
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school BtudftntB from the tracked and nontracked high schools. For 
the years 1978, 1979, and 1980, a greater proportion of tracked 
black Atudents passed than did non-tracked blacks (520 F.Supp. at 
484) . Following this conflicting evidence, the judge wrote ^1 
find it inpossible to accurately evaluate the effects of 
achievexBent grouping** (520 F.Supp. at 485). 
2.4.1.3 Dillon County 

The Office of Civil Rights, United States Department of 
Education (OCR) initiated action against the Dillon County School 
District Lakeview, South Carolina alleging that the district was 
in violation of Title VI of the Civil Rights Act of 1964 because 
of the district *s failure to alter or abandon its practice of 
assigning students in grades 1-8 to racially identifiable ability 
grouped classes, without educational justification. The OCR 
further alleged tliat compliance by the district with Title VI 
could not be achieved by voluntary means and thus sought an order 
of relief terminating and refusing to grant or continue further 
federal financial assistance to the district until the district 
could satisfy the OCR that it h&d appropriately remedied Its 
alleged failure to assign students on the basis of educationally 
valid, non-discriminatory procedures and that it would comply in 
the future with all applicable requirements of Title VI of the 
Civil Rights Act of 1964. The district denied the allegations of 
the OCR and claimed that its student assignment methods are in no 
way motivated by racially discriminatory intent but rather by the 
purpose of providing better educational opportunities. The 
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failure of OCR and the district to reach a voluntary agreement 
led to an adalnlstratlve hearing, In the UoS. Department of 
Education, (docket number 84-VI-16, compliance proceeding under 
Title VI of the Civil Rights Act of 1964, Eugene Powell, Jr. 
presiding judge) . 

Starting In school year 1982-83, the district began 
assigning students to regular ability-grouped classes In grades 
1-8 by relying primarily on the functional equivalent of a rank 
ordering of the Comprehensive Test of Basic Skills (CTBS) scores. 
The district employs the "block placement** approach In grades i-s 
with the CTBS total battery scores a primary criterion for 
placement. Students In grades 1-6 are i^laced by the block method 
within self-contained ability levels for the entire day for all 
subject areas with the exception of those students who are pulled 
out of the classroom for Chapter l remediation In either reading 
or mathematics, students In grades 7 and 8 are assigned to 
departmentalized classes whereby they move together as a block 
from subject to subject for the entire academic day. The 
kindergarten and high school placement methods are exceptions to 
the block placement method of assignment. 

The result of this ability grouping on the basis of CTBS 
total score Is a large number of **raclally Identifiable** 
classrooms; for the school year 1983-84, 19 out of the 34 classes 
In kindergarten through eighth grade were racially Identifiable. 
The criteria for racial Identlf lability Is the 20% **rule of 
thumb** or »*trlgger** used for many years In discrimination cases 
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by nany govammdnt agencltts as an Initial prasumptlon of adverse 
discriminatory Impact e The dasj Is judged **racially 
Identifiable" when the black percentage composition of a class 
differs by more then twenty percentage points from the overall or 
average black percentage composition of the grade as a whole. 
Moreover, the top ability levels always contain the lowest 
percentage of blacks, and the lower ability levels contain the 
highest percentage of blacks. These results were presented as 
highly statistically significant and practically Important. 

Two Important types of questions about student academic 
progress were prominent In the hearing: 1. Student mobility 
between tracks over time and, 2. The academic progress over time 
of students in the lower track. Mobility between tracks can be 
examined by the analysis of transition matrices for adjacent 
years, such as the transition matrix Figure 1, or more 
comprehensively by multivariate versions of the transition matrix 
for more than two years. Of particular importance is whether the 
transition matrices (and therefore specific transition 
probabilities) differ for white and black students. An analysis 
of mobility between tracks presented In the administrative 
hearing revealed a ''pattern that black students tend to move down 
in the ability groups and white students tend to move up. If the 
white student does move down in ability level, he or she tends to 
bounce right back up. There is only a 17% likelihood that whites 
will move down from the top track while there is a 39% percent 
likelihood that they will move up from the bottom track. The 
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pattern is exactly reversed for blacks In that 31% of the time 
blacks moved down when they had reached the top level and 21% of 
the tine they nove up from the bottom track. There is a similar 
pattern in analyzing the middle track although this pattern is 
not as dramatic as the analysis of the top and bottom tracks** 
(pp. 53-54 of Administrative Proceedings) . Differential mobility 
among ability groups or tracks is an important general issue, and 
one which needs considerable methodological and statistical 
attention. 



Insert Figure 1 here 



A second issue is the effect of the ability grouping on 
student progress. An analysis presented by OCR centered on 
students with the same or matched subtest score (e.g. math) but 
who were in different tracks (track placement was determined by 
the overall battery score). Examining those pairs of students* 
subsequent achievement scores for that subtest indicates whether 
there is differential progress in that subtest area. A 
discernable pattern was found in the relationship between 
placement in a track and subsequent acr.demic progress and 
performance. In some, being placed in a relatively high track 
leads to relatively better performance while being placed in a 
relatively low track would seem to lead to relatively poor 
performance. The ability group level in which a student is 
placed, even though he or she begins with the same subtest score, 
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Figure 1. Transition matrix showing mobility between seventh and eighth 
grade ability groupings (1 high, 4 low) for Black (B) and White (W) 
students . 
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dlrsctly correlates with whether he or she will do better or 
worse on a test the following year. Being placed in a low track 
is to the student's detriment compared to placement in a high 
track. These findings have statistical and practical 
significance and show that although students may start out with 
roughly comparable scores, by the end of the year, the placement 
in a different track has increased the gap between them (pages 
54-55 of the Administrative Hearing) . The district attempted to 
use evidence of academic progress to find an educational 
justification for the ability grouping; favorable statewide 
testing results and a 2% annual increase (in percentile rank 
metric) among the lower groups in almost ail grades were part of 
the district's defense of the educational justification of its 
ability grouping. 

Another important issue related to ability grouping that was 
prominent in this case is the superiority of bivariate or 
multivariate classification as opposed to univariate 
classification. By univariate classification we mean practices 
similar to the Dillon County District, where a single score, 
composed of the aggregated math, reading, and language subtes 
scores, is used to assign a student to a certain ability group 
and that student assignment stays the same in all the relevant 
subjects. A multivariate classification would use the relevant 
subtest score to group students for that domain; e.g., students 
are grouped by their mathematics ability (based on the 
mathematics subtest scores) for their mathematics classes, by 
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their reading alsility for reading classes. This aultivariate 
classification would; for exanple, allow a student to be in the 
high group on mathenatics and perhaps in the lower group on 
reading « Testimony regarding the bivariate classification for 
math and reading led the judge to conclude that ability grouping 
done for m^th and reading on the basis of the respective subtest 
score with the remainder of the subjects heterogeneously grouped 

» 

by random method would result in approximately 32% (or using the 
time weighted theory, 23%) of the classes being racially 
identifiable (p. 74 of the Administrative Proceeding) . This is 
far superior than the 52% of racially identifiable classes with 
present univariate grouping. 

As a result of the evidence presented on racial 
identiflability resulting from ability grouping, the diTtferential 
mobility between blacks and whites, and differential academic 
progress and achievement as a result of ability grouping the 
administrative law judge concluded that Dillon County is in 
violation of Title VI of the civil Rights Act and that the U. S. 
Department of Education has authority to terminate all federal 
financial assistance under Chapter 1 and Chapter 2 of the 
Educational Consolidation and Improvement Act of 1981. 
2.4.2 Legislation and Litigation Regarding Special Education 

Progress of special education for handicapped students in 
alternative school settings has been prominent in many judicial 
decisions. The Education for All Handicapped Children Act of 1975 
(Public Law 94-142, 89 Stat. 773) assures all handicapped 

?-41 



children the right to a free and appropriate education. **It is 
the purpose of this Act to assure that all handicapped children 
have available to them ... a free appropriate public education 
which emphatilzes special education and related services designed 
to meet their unique needs . . . and to assess and assure the 
effectiveness of efforts to educate handicapped children" (&9 
Stat, at 775). One way to assess the effectiveness of efforts to 
educate handicapped children Is to provide an Individualized 
educational program (IE?) for each child. *«The term 
* Individualized education program' means a written statement for 
each handicapped child** which shall include at least the 
following: **a statement of the present levels of educational 
performance of such child, a statement of annual goals, . . . 
appropriate objective criteria and evaluation procedures and 
schedules for determining, on at least an annual basis, whether 
instructional objectives are being achieved** (89 Stat, at 776) . 
The lEP must Include Initial status, the instructional goals, and 
a method to monitor the student *s progress toward meeting these 
goals. These lEPs are to be revised at least annually: once a 
year, progress is to be assessed and new objectives outlined. The 
details of the use and formulation of lEPs has been discussed in 
the section on data and methods for assessing student progress. 
2.4.2.1. Special Education Issues in Tattnall County Case 

Tattnall County places a larger proportion of its students 
into special classes than does the srate of Georgia or the 
nation. "Placement in the EMR program for a student who Is not 
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truly retarded is disastrous .... Ths longer the child Is 
mlsclasslfled, the nore severe and difficult to repair the damage 
caused by the nlsclasslflcatlon becomes** (520 F.Supp. at 495). 

The state of Georgia regulates the procedures for placement 
Into special classes. A student Is first referred for evaluation, 
then Is given a physical examination, and then psychological 
tests. A Special Education Placement Comm^.ttee Is to decide on 
the appropriate program for the child. An Individual educational 
program is drafted and revised annually. The placement decision 
is to be reevaluated every three years. Parental consent is 
necessary before the student may be placed in a special program. 

In Tattnall County, a **grossly disproportionate number of 
black children were classified as mentally retarded while the 
classes for children with specific learning disabilities were 
mostly white" (520 F.Supp. at 496) « The disproportionate racial 
Impact implied that some children were misplaced. Examples were 
provided of several students who were placed in EMR classes yet 
had IQ or CAT scores well above the cutoff for EMR placement. It 
was discussed that average students are expected to gain about a 
year in achievement (presumably, a point in grade-equivalence 
scores) for each chronological year, whereas slower students are 
expected to gain at slower rates (520 F.Supp. at 497-98). 

In regard to the evident mlsclassification of students into 
special classes, the Court requested that the plaintiffs and 
defendants submit a plan to the court to provide remediation for 
misclassif led students and a plan for a reevaluation of all 
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students class 1 fed as EMR. 

2.4.2.2. Hoffman v> Board of Education of the Cltv of New York 

The background for this educational aalpractlce case 410 
N.Y.S.2d 99 (1978), was that Hoffman scored 74 on an IQ test In 
first grade and was placed In a class for •'Children with Retarded 
Mental Development (CNRD) . He had a speech Inpedlnent (since his 
father's death) and was taken to a speech disorder clinic at 
about the same time. At the time of the IQ testing, the school 
psychologist recommended that Hoffman be "reevaluated" within two 
years, since his IQ score was virtually on the cutoff for special 
class assignment. Hoffman remained In CRMD classes until the 12th 
grade, at which time all those students were transferred to an 
Occupational Center for the handicapped. To verify eligibility 
for this occupational program (I.e., mentally handicapped), all 
the students were administered IQ tests. Hoffman scored In the 
90s— well within "normal" range. He was therefore disqualified 
from attending this vocational school. 

Hoffman sued the school district for educational 
malpractice. His claim Included digging up the score on a non- 
verbal IQ test given to him at the Speech Disorder Clinic twelve 
years prior. He had scored In the 90s then, and claimed that he 
should not nave been retained In the special classes for his 
entire schcni career. Rather, the school district had the 
obligation to reevaluate him within the two years after Initial 
placement, as the school psychologist had recommended. To the 
plaintiffs, "reevaluate" meant "retest." 

2-44 

9.. 118 



The defense claimed that "reevaluate** meant just that— and 
that all students are reevaluated continuously by their teachers. 
In addition, district-wide achievement tests are administered 
twice yearly, and Hoffman's scores were ••still well below what 
would be expected of a child in a regular class^^ (410 N.Y.S.2d at 
103). The defendant claimed that, had the boy^s scores been 
higher, he would have been retested on ability and possibly 
placed into regular classes. They did not contest the boy's 
"true" IQ of 96. 

Hoffman prevailed in the District Court, but the decision 
was reversed in appeals court. The appeals court did st^te that, 
"Fortunately, since 1968 no such error is likely to occur for 
both New York State and the New York City Board of Education now 
require frequent and periodic intelligence retesting of children 
in CRMD classes" (410 N.Y.S.2d at 108). 

The issue of general concern is that Hoffman was a child of 
normal intelligence who was erroneously placed in the CRMD 
classes. Could we expect him to have high enough achievement 
scores (on the semi-annual tests) to signal the administrators to 
retest (IQ) him? How much is a child of normal intelligence 
expected to learn in a restricted learning environment? How can 
this progress properly be assessed? 
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SECTION 3 
CRITIQUE AND RE ANALYSIS 
The reanalyses in this section illustrate certain types of 
statistical analyses of student progress^ which are based on 
Rogosa^s prior technical research. Detailed attention is given 
to the special issues arising with data from educational court 
cases. The two sets of data we were able to obtain for this 
project were from the Tattnall County case (^.ections 2.2.3 
2.^.1.2 and 2.^.2.1) and from Scheelhaase v. Uoodburv (section 
2.1.1). The reanalyses allow both a critique of the kinds of 
data and data analyses used in the court cases and also 
demonstrations of some improvements. In section ^ the details of 
improved analyses of student progress are further explained • 
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3.1 TATTNALL COUNTY DATA REANALYSIS 

Access to the raw data upon which some of the plaintiff's 
testimony was based allowed detailed examination of the quality 
of the data in the court case and our independent reanalyses. We 
describe the available data, our descriptive analyses of student 
progress, and statistical estimation of curricular correlates of 
student progress. Particular attention is given to the metric 
<and internal consistency) of the achievement test data 
including a comparison of statistical analyses based -n i,rade 
equivalents and raw scores. These data are of particular 
interest because of the prominent role of questions about 
student progress in the testimony and the court's decision. 
3.1.1 The Data 

The data which are used in these r ilyses were supplied to 
Dr. Robert Calfee (Stanford University, School of Education) 
during the course of his involvement with the Tattnall County 
cases (sections 8.2.3 and 2.^.1.2). The data consist of scores 
and background information for a cohort of Tattnall County high 
school students who were freshmen in 1976 and who were tested 
yearly until graduation in 1979. 

The data analysis files come from two sources: <1) a xerox 
listing of reading and mathematics raw scores and grade 
equivalents (GE*s) for 212 students, r.nd <2) a computer file 
(called GAREC) created for Dr. Calfee's analyses of a sample of 
50 students. The latter file also contains background data on 
each of the 50 students as well as numerous variables created 
through Dr. Calfee's analyses. Calfee's analyses are described 



and evaluated by the court in Anderson v. Banks 5^0 F.Supp. 761 
(19B2, pp. 762-5). 

A three-^digit identification number e>^isted for each 
student on the xerox listing (n * SIS) and codes were also 
present for "race," "gender" and "school attended." Gender was 
used as a background variable in early analyses, before GAREC 
was located. From the variables on GAREC, four were selected as 
most promising for subsequent work. These were ELEMREAD, 
ELEMMATH, FINEENG and FINEMATH. ELEMREAD and ELEMMATH are 
identified as "GR ^-8 GPA [average grade point average in the 
subject over grades ^ through 83" for reading and mathematics, 
respectively. They are coded as whole numbers, 1 through ^. 
FINEENG is a "fine curriculum index" in english and FINEMATH is 
a similar index for mathematics. From Dr. Calfee, we were able 
to ascertain that they are coded so that higher values reflect 
more c?xposure to regular courses, and lower values correspond to 
more remedial courses. 

It will be noted that only the grade equivalent scores 
appear in GAREC. However, we were able to associate background 
variables with raw scores by matching student identification 
numbers and merging the appropriate information from hard copy 
and tape files. 

The California Achievement Tests (CAT) were used for the 
testing in Tattnall County <see discussion of CAT in Anderson v. 
Ba nks and Johnson v. Sikes 520 F.Supp. ^72 <19B1, pp. 485-94). 
However > the precise edition, level and form were not specified 
in any of the historical records made available to us. This 
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became important as some of the data were incomplete and 
% apparent inconsistencies existed in those data that were present 

(see also the court's discussion SSO F.Supp. at ^9S). 

After considerable investigation and analysis it appears 

# that the 1970 edition of CAT, Level 5 Form A (CAT/5A) (and 
possibly Form B as well, though this has not be verified) was 
used for the 1976, 1977 and 1978 testings. The 1977 edition of 

# CATf Level 19 Form D (CAT/19D) appears to have been used for the 
1979 testing of the students as 12th graders. Table 1 shows the 
maximum raw score? the maximum GE, and the raw score at which 

# the maximum GE is first attained for each of the tests. 

I nser t Tab 1 e 1 her e 
Descriptive analyses. The full data set consisted of 

# records for 212 individuals. However, not all individuals had 
complete records of test scores at all four administrations. 
Table 2 shows means and standard deviations of raw scores and 

S grade equivalents for those students who had complete testing 

information in reading and in mathematics. (Inexplicably, some 
students had GE's when no raw score appeared on the file. 

9 Consequently, there are more students with complete GE records.) 

Note that the GE's increase each year in magnitude and maintain 
roughly the same variance. However, the raw scores in both 

9 reading and mathematics show a drop in Grade 12 (recall that 

there tire fewer items on CAT/19D than on CAT/5A). 

Insert Table 2 here 

§ 3.1.2 Initial Statistical Estimation of Growth 

For the first set of analyses, we take the date simply "as 
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Section 3.1 



TABLE 1 

CHARACTERISTICS OF THE TESTS 



TEST Max Raw Max GE Raw Score at which 
Score . Max GE is attained 

^ CAT/5A Reading Total 85 13.6 67 

CAT/5A Mathematics Total 98 13.6 77 

CAT/19D Reading Total 70 12.9 46 

• CAT/19D Mathematics Total 85 12.5 53 
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TABLE 2 

MEANS AND STANDARD DEVIATIONS 
TATTNALL COUNTY FULL DATA 

READING: 

RAW SCORE CRAPE EQUIVALENT 

Mean S.D. Mean S.D. 

(n - 147) (n - 152) 



36.9 


12.7 


7.7 


2.6 


39.5 


14.5 


8.6 


2.9 


42.2 


14.7 


9.1 


2*8 


41.3 


12.8 


10.0 


2.1 




MATHEMATICS: 




RAW SCORE 




(SHADE EQUIVALENT 


Mean 


S.D. 


Mean 


S.D. 


<n - 146) 




(n - 152) 




42.3 


18.4 


7.8 


2.6 


50.5 


19.6 


9.1 


2.8 


56.7 


17.7 


10.0 


2.6 


50.2 


14.3 


10.7 


1.9 
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is** except for arbitrarily supplying a constant valuer zeroy as 
the decimal value in the grade IS reading GE« <Apparently a 
similar approach was taken v^ith the analyses presented to the 
courts since the ISth grade reading GE's in GAREC all ended i^ith 

Then individual growth curves are fit to each student's 
data. Analyses are done separately for Reading Total raw 
scores^ Reading Total GE'sy Mathematics Total raw scores and 
Mathematics Total GE's. 

Since only four points in time are available <the testings 
in grades 9f lOf 11 and IS) f straight-line growth curves are fit 
using ordinary least squares. Thusy the model for person p is 



X (t) « { <0) « t -4- « 
P P P 



wher< 



X (t) is the observed score for person p at time t f 
p 

Cp<t) is the true score for person p at time t » 
Op is the true linear growth rate of person p y 

t is time and 

t is random errors assumed to be independent at each 

time. 

Fitting this model allows the estimation of a constant rate of 

2 

change for each student. Ue also examine the R values for 

each fit to get some idea of how adequate a straight-line model 

^ 2 

is for these data* Five number summaries of » and R are 
shown for each of the four analyses in Table 3. It appears that 
grade equivalent scores give better results than raw scores. 
Note the median R is 60.^ and 63.3 fo.^ reading and math 6E's> 
respectively; compare this with 33.6 and S5.1 for reading and 
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mathematics raw scores^ respectively. Also note that the median 
rate of change is slightly less than one GE per year in both 
reading and mathematic9>. Statistical proocedures for this 
estimation have been implemented for this project in SAS for 
mainframe computers and in the language GAUSS for MS-DOS 
microcomputers) an appendix to this report contains the listings 
of these programs. 

Insert Table 3 here 

Table 4 shoves estimates^ computed from the Tattnall County 

datay of a number of these quantities v^hich describe properties 

of a collection of individual growth curves.. To briefly 

2 

summarize^ t® and k characterize the time scale) <r amd 
describe the variance in growth rate and status; P(b) 

fit'') 

/\ 

and the standard error of o characterize the reliability and 

precision of the estimates of straight-line growth rate? y is 

an index of tracking that describes the stability of individual 

differences over time) and p^^^ is the correlation between 

growth rate and some background variable^ U . For more detailed 

descriptions of these quantities^ see Rogosa and Uillett (1965)^ 

Insert Table ^ here 

Tables 5 and 6 give some indication of the consistency of 

high and low scoring students. The student identification 

numbers are listed for those students with the five lowest and 

five highest values of the variables shown. For example^ note 

2 

that student #109 has the second highest value of R among the 
fits for Reading Total raw score and the third highest on 
Mathematics Total raw score. Not surprisingly » this student has 
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TABLE 3 

FIVE NUMBER SUHMAKIES OF EMPIRICAL RATE AND 
TATTNALL COUNTY PULL DATA 



READING 

• RAW SCORE QRAPE EOVmLSNT 



#147 


FATE 




#152 






M 


1.7 




M 


0.7 




F 


-0.2 


3.5 


F 


0.4 


1.1 


5% 


-2.9 


5.7 


5% 


"0.4 


1.8 


* 


-14.2 


18.7 


* 


-1.4 


2.9 


#147 


r2 




#152 


r2 




M 


33.6 




N 


60.4 




F 


10.9 


66.9 


F 


24.5 


84.9 


5% 


0.4 


92.7 


5% 


2.6 


97.4 




0 


100.0 


* 


0 


99.5 



RAW scokr 



#146 




RATE 


M 




3.0 


F 


0.5 


5.7 


5% 


-4.1 


9.8 




-11.0 


12.6 


#146 




r2 


M 




25.1 


F 


6.8 


63.1 


5% 


1.0 


95.2 




0.0 


98.0 



MATHEMATICS 



GRADE EQUIVALENT 



#152 




BAT£ 


M 




0.9 


F 


0.6 


1.4 


5% 


-0.3 


2.0 


* 


-0.9 


2.3 


#152 




r2 


M 




63.3 


F 


37.7 


88*6 


5% 


2.5 


97.3 


* 


0.1 


98.9 



f 
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ESTIMATE 



TABLE 4 

STATISTICAL AND PSYCHOMETRIC QUANTITIES 
TATTNALL COUNTY FULL DATA 

ItEADING 

RAW SCORE 



GRADE EQUIVALENT 



t 

K 

2 

P(e) 

A 

s.e. (e) 



ew 



10.796 
6.375 
3.441 
139.882 
.281 
2.964 
.766 
-.085 



15.417 
5.787 
.089 
2.978 
.217 
.567 
.791 
.009 



i 

.2 



K 

P(e) 

A 

S.e. (e) 

Y 

Pew 



MATHEMATICS 
RAW SCOR E* 



.769 



GRADE EQUIVALENT 
18.232 
.865 
.077 
.057 
.201 
.552 
.777 
-.223 



Note: W is dichotomous gender variable. 

* Warning: High SSRES causes computational error in some of these 
quantities. 
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the fourth lowest residual variance on reading. Student #109 

has the highest learning rate in mathematics and the second 
/\ 

lowest i (which indicates that their relative position in the 
P 

group is changing a lot over time). 

Insert Tables 5 and 6 here 

Tables 7 through 11 present similar analyses for the subset 
of 50 students that were used for analyses presented in Anderson 
V . Banks . In the following these 50 cases will be referred to 
as the **subset data.** Table 7 shows that grade equivalents 
increase each year whereas raw scores increase until grade IS 
and then drop. Table 8 showsy again^ that GE's appear to work 
better for a straight-line prowth model with these data. Tables 
9« lOy and 11 repeat the analyses shown for the full data in 
Tables 4> 5 and 6> respectively. Gender was used for the 
background variable^ in the computation of P « 
3«1.3 Additional Growth Curve Analyses 

During the completion of these analyses^ several issues 
arose which gave rise to the subsequent analyses. Firsts it was 
recognized that it would be desirable to incorporate other more 
relevant background variab7»es in the analyses if such variables 
could be identified. Second^ it was realized that there existed 
considerable potential for topping out when using GE scores 
since the maximum raw score necessary to achieve the highest GE 
is relatively low (see Table 1). Related to this concern is the 
more general issue of which score metric is most useful for 
describing growth. Final ly^ it was beginning to become clear 
that the data were not consistent with the hypothesis that all 
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TABLE 5 
TATTNALL COUNTY FULL DATA 
CASES WITH EXTREME VALUES OF SELECTED QUANTITIES 

RAW SCORES 



0 



READING 
r2 



Y 



034 012 057 017 

196 144 042 034 

149 108 074 093 

^ 131 128 109 149 

• 171 089 032 172 

052 074 190 195 

048 093 075 193 

063 021 002 188 

093 109 196 192 

• 017 057 017 187 



LOW 



HIGH 



MATHEMATICS 



e r2 



196 019 171 152 

152 161 028 109 

167 184 086 097 

052 038 062 052 

183 090 041 158 

089 094 036 168 

149 028 047 178 

075 109 119 192 

097 123 196 012 

109 063 026 161 



i 



ft 



i 



i 
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TABLE 6 

• TATTNALL COUNTY FULL DATA 

CASES WITH EXTREME VALUES OF SELECTED QUANTITIES 

GRADE EQUIVALENTS 



READING MATHEMATICS 



i 



A 

e 


R2 


e 


A 

Y 




A 

e 


R^ 


e 


A 

Y 


196 


061 


045 


093 


LOW 


196 


144 


081 


040 


149 


189 


064 


149 




152 


147 


111 


097 


131 


Oil 


052 


048 




161 


128 


017 


108 


192 


183 


030 


196 




150 


158 


019 


152 


187 


139 


040 


131 




192 


019 


054 


089 


018 


054 


087 


185 




063 


044 


188 


003 


023 


064 


138 


195 




089 


111 


117 


004 


101 


030 


Oil 


159 




116 


041 


029 


028 


048 


045 


075 


193 




097 


014 


125 


001 


093 


052 


002 


003 


HIGH 


040 


017 


026 


012 



i 



i 
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four testings had been done with the same test (as originally 
believed). Thus the data should perhaps not be taken "as is" 
for the analytical purpose of estimating growth. Subsequent 
analyses attempted to deal with these issues. 



3.1.3.1 Curricular Variables 

First, an attempt was made to identify other suitable 
background variables for analysis. This was accomplished when 
(after considerable detective work) the tape file GAREC was 
located and investigated. As explained earlier, the variables 
ELEMREAD and FINEENG were selected as background variables for 
Reading Total and ELEMMATH and FINEMATH were chosen for use with 
Mathematics Total. Since the GE's had appeared most useful in 
the first analyses, the grade equivalent analyses were repeated 
using the GE values recorded in GAREC (these were the same as 
before except, as noted, that the grade 12 Reading Total GE*s 
arbitrarily had been assigned a four to the right of the decimal 
rather than a zero). 

For these analyses, attention was focused on the correlates 
of change. Systematic individual differences in growth were 
modeled by 



where W stands for an exogenous background variable. See Rogosa 
and Willett (1985) for a complete description of this model and 



its properties. Table 12 shows the estimated values of p^^^ and 
several other quantities of interest. It will be noted that 
p ranges from -.^7 to -.63 . These values were contrary to 



Insert Tables 7 through 11 here 



E(B|W) - * fi^^^(W - H^) 
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TABLE 7 

• MEANS AND STANDARD DEVIATIONS 

TATTNALL COUNTY SUBSET DATA 

READING 

RAW SCORE GRADE EQUIVALENT 

• Mean S.D. Mean S.D. 
GRADE (n - 47) (n - 50) 

9 34.4 12.7 7.3 2.7 

10 34.6 14.4 7.7 2.8 

11 37.1 13.3 8.4 2.8 
H 12 36.3 12.1 9.4 2.2 



MATHEMATICS 



RAW SCORE GRADE EQUIVALENT 

i Mean S.D. Mean S.D. 

GRADE (n - 49) (n - 50) 

9 36.9 18.3 7.2 2.6 

10 44.5 20.2 8.5 3.0 

11 52.3 19.0 9.6 2.8 
i 12 44.8 15.2 10.0 2.2 



i 



i 



i 
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TABLE 8 

FIVE NUMBER SUMMARIES OF EMPIRICAL RATE MD R^ 
TATTNALL COUNTY SUBSET DATA 

RAW SCORE GT^ADB EOUIVALENT 





RATE 




#50 




RATE 




1.2 




M 




0.8 


-1.0 




3.5 


F 


0.2 


1.1 


-8.8 




5.7 


5% 


"0.5 


1.9 


-14.2 




6.1 


* 


-1.4 


2.1 




r2 




#50 








40.0 




M 




60.4 


11.7 




65.3 


F 


29.5 


81.7 


0.4 




95.3 


5% 


0.3 


98.7 


0 




100.0 


* 


0 


99.5 



MATHEMATICS 

RAW SCORE GRADE EOUIVALENT 

RATE #50 Em 

3.2 M 1.0 

0.8 5.7 F 0.7 1.3 

•4.1 10.2 5% -0.2 2.0 

•11.0 12.6 * -0.9 2.3 



Ri #50 

3.2 M 59.6 

6.9 58.4 F 38.1 84.4 

1.1 94.8 5% 3.2 97.7 

0.02 97.5 * 1'2 98.6 
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TABLE 9 

STATISTICAL AND PSYCHOMETRIC QUANTITIES 
TATTNALL COUNTY SUBSET DATA 



ESTIMATE 
to 

K 

s.e. (e) 

Y 

Pew 



READING 

RAW SCORE 

11.002 
4.586 
5.472 
115.096 
.372 
3.037 
.742 
.255 



GRADE EQUIVALENT 

12.337 
4.969 

.180 
4.439 

.372 

.551 

.762 

.237 



K 

S.e. (e) 

Y 

Pew 



MATHEMATICS 
RAW SCORE* 



.788 



GRADE EQUIVALENT 

22.615 
4.992 
.031 
.769 
.091 
.555 
.799 
.496 



Note: H is dlchotonous gender variable 

* Warning: High SSRES causes computational error In Bome of these 
quantities. 
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TABLE 10 
TATTNALL COUNTY SUBSET DATA 
CASES WITH EXTREME VALUES OP SELECTED QUANTITIES 



RAW SCORES 



MATHEMATICS 



A 

0 


r2 




A 

y 






r2 


^ 2 


A 

Y 


034 


012 


057 


034 


LOW 


196 


019 


041 


109 


196 


089 


042 


048 




052 


057 


025 


089 


131 


027 


109 


106 




096 


189 


109 


052 


061 


138 


032 


023 




131 


138 


001 


010 


025 


037 


040 


013 




043 


043 


009 


096 


106 


052 


194 


155 




180 


009 


061 


159 


163 


163 


087 


159 




010 


014 


045 


012 


180 


106 


138 


163 




040 


025 


036 


054 


052 


109 


034 


180 




089 


041 


196 


023 


048 


057 


196 


194 


HIGH 


109 


109 


026 


189 
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TABLE 11 
TATTNALL COUNTY SUBSET DATA 
CASES WITH EXTREME VALUES OF SELECTED QUANTITIES 

GRADE EQUIVALENTS 



READING 



e 



196 161 045 048 

131 189 052 060 

194 005 040 106 

013 014 054 013 

034 057 009 034 

052 048 060 163 

106 040 027 180 

060 054 005 194 

023 045 087 159 

048 052 138 189 



MATHEMATICS 

e r2 a| Y 



LOW 



HIGH 



196 019 019 040 

189 134 054 089 

134 052 014 010 

019 096 041 043 

096 013 189 096 

014 054 180 137 

109 109 027 163 

010 010 194 012 

089 041 163 023 

040 014 026 060 



'eric 



13S 



expectation since they implied ^ for example^ that more exposure 
to \ ^gular courses i^as associated with slower growth. It was 
suggested that perhaps these values were artifacts of the 
ceiling effect of the GE's. That is> students with high values 
of FINEENG and FINEMATH perhaps have higher values of GE to 
start with and thus can't exhibit much growth due to the low 
ceiling of the GE scores. 

Insert Table i2 here 
Analyses on Imputed Raw Scores. 

Parallel to this investigat ion^ we decided to try to 
reappraise^ in the context of Tattnall County's data^ the 
relative merits of raw scores and GE's for the purpose of 
studying academic growth. Also* as mentioned earlieri we needed 
to identify precisely which edition^ form and level of the CAT 
had been used for each testing. 

The plots from Section 1.3 show the form of the 
transformation of raw score into Grade Equivalemts for each 
grade and for each of Reading Total and Mathematics Total taken 
from the norms tables of the 1970 CAT/5A and the 1977 CAT/19C 
and CAT/19D. Figures 1-8 of this section show the "empirical" 
GE-raw score correspondence for the test scores of the 
individuals in the Tattnall County subset data. The first four 
plots show the reading scores and the next four the mathematics 
scores. After carefully comparing the normative plots with the 
empirical plots* checking the empirical data against both sets 
of norms tables and noting the percentage of agreements and 
disagreements^ it was concluded that the first three testings 
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TABLE 12 

CORREIATES OF CHANGE 
TATTNALL COUNTY SUBSET DATA 
GRADE EQUIVALENTS 



READING TOTAL WITH ELEHREAD OR FINEENG 
ESTIMATE ELEMREAD FINEENG 

PqW -.587 -.527 

-.344 -.026 



Pew 



.495 .568 

1.378 .131 

MATHEMATICS TOTAL WITH ELEMMATH OR FINEMATH 

ELEMREAD FINEENG 

-.631 -.468 

-.216 -.014 

.372 .213 
.648 .032 
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most probably belonged to CAT jA while the grade 12 testing 
appeared to be CAT/19D. 



Insert Figures 1-8 here 



Imputation 1 > In order to avoid the truncated high end of 
^ the GE scale but still maintain some degree of comparability 

across the four years » it was decided to use the raw scores from 
the first three years and to impute a comparable raw score for 

0 the twelfth grade by replacing the CAT/19D raw score with the 
CAT/5A raw score which corresponded to the same GE. For the top 
GE of the scales the raw score at the midpoint of the 

^ corresponding range of raw scores was imputed. Table 13 shows 

that when this is done^ the large negative correlations with he 
curriculum indices disappear. That is» with grade equivalents 

^ correlations with the curricula indices <R>M) are < -.5S7f-.^6Q) 

from Table 12> whereas these correlations become near zero for 
raw scores in Table 13. 

§ Insert Table 13 here 

Imoutation 5. In an effort to further refine the raw 
scoresy two other types of imputation were tried. The next type 

9 (Imputation 2) imputes raw scores from matching 6E'*^i exactly 

where possible just as in Imputation 1 described above. The 
difference lies in how we handled the top end of the GE scale. 

1 Imputation 2 used the norming table GE's and raw score© from the 
middle range of the table to extrapolate a raw score-GE 
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TABLE 13 

CORRELATES OF CHANGE 
TATTNALL COUNTY SUBSET DATA WITH IMPUTED RAW SCORES 



ESTIMATE 

Pew 

^ew 

Pc(tO)W 



^ew 
^ew 



READING 

ELEMREAD FINEENG 

-.412 -.097 

-1.017 -.020 

.753 .754 

11.733 .975 

MATHEMATICS 

ELEMREAD FINEENG 

-.398 -.032 

-1.323 -.009 

.628 .478 

11.552 .740 



Note: Imputation 1 used 
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correspondence for the high end of the scale. Then twelfth 
grade raw scores were imputed by matching the predicted GE 
values of CAT/19D with those of CAT/5A. 

Imputation 3. Imputation 3 was a different approach. 
Rather than use the GE's to achieve the imputationy we simply 
calculated percent-correct scores on CAT/SA and CAT/19D by 
dividing the scores on each by the respective total number of 
items. Then the raw score for CAT/19D was imputed as the CAT/5A 
raw score that corresponded to the same percent-correct score. 

Table 14 compares the values of p for the four sets of 

analyses (GE's and the three types of imputed raw scores). As 

can be seen, the negative correlations with FINEENG and FINEMATH 

disappear when raw scores are used, and the correlations with 

ELEMREAD and ELEMMATH decrease in magnitude except for 

>% 

ElEMMATH, re^w score Imputation 3 where p goes to -u705! 

Insert Table 14 here 
Table 15 compares the five number summaries of the squared 
multiple correlation for the individual growth curve fits (RSQ) 
for each of these four analyses. For Mathematics, raw score 
Imputations 1 and S clearly produce better fits than using the 
GE's while raw score Imputation 3 does not do as well. For 
Reading Total, it appears that the GE's conform slightly better 
to the straight-line growth model than do any of the raw score 
imputations . 

Insert Table 15 here 
3.1.4 Summary and conclusions 
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TABLE 14 

COMPARISON OF ESTIMATED CORRELATES OF 
RATE FROM GE AND RAW SCORE ANALYSES 



Section 3.1 



READING 
GE 

RS: Imputation 1 
Imputation 2 
Imputation 3 

MATHEMATICS 
GE 

RS: Imputation 1 
Imputation 2 
Imputation 3 



ELEMREAD 

-.587 

-.416 

-.581 

-.305 

ELEHMATH 

-.631 

-.398 

-.344 

-.705 



FINEENG 

-.527 

-.097 

-.151 

-.107 

FINEMATH 
-.468 
-.032 
+ .124 
+ .052 
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TABLE 15 

FIVE NUMBER SUMMARIES OF R^ 
FOR ANALYSIS OF TATTNALL COUNTY SUBSET DATA 



GE 



#43 

M 

F 

5% 

* 



READING 



65.2 

37.7 88.4 

.4 98.5 

0. 99.0 



#47 

M 

F 

5% 

* 



MATHEMATICS 



60 



43.7 
2.3 
1.2 



86.5 
97.9 
98.6 



RS Imputation 



RS Imputation 



RS Imputation 



1 #40 
H 

F 

5% 
* 

2 #40 
M 

F 

5% 

* 

3 #40 
M 

F 

5% 

* 



59.5 

33.0 78.9 

1.9 97.9 

.9 98.8 



59.5 

33.0 85.0 

1.9 97.9 

.9 98.8 



53.7 

22.4 80.1 

1.5 98.4 

.2 99.5 



#42 

M 

F 

5% 

* 

#42 

M 

F 

5% 

* 

#42 

M 

F 

5% 

* 



75.0 

49.8 93.3 

7.6 98.3 

3.5 99.5 



75.0 

49.8 92.2 

7.6 98.3 

3.5 98.5 



55.7 

29.5 77.9 

.9 99.5 

.2 99.7 
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Analyses based on statistical estimation of growth curves 

^ were applied to the achievement data from the Tattnall County 

cases. As with many reanalyses perhaps more questions were 
raised than resolved. Clearly, the data and their analyses for 

1^ the court case have myriad shortcomings; yet, these problems are 

even more profound in the other cases we examined. But none of 
the flaws in data and data analysis vitiate the central role of 

^ questions about student progress. 

Qualitv of data. Although the data on student achievement 
and curricula are more extensive than in other cases, the 

^ serious flaws and ambiguities in these data render any 

inferences from their analyses open to debate. Even the court's 
statement about the actual tests that were administered <CAT 

^ 1970 and 1977 editions) are not totally consistent with the 

numerical values of raw scores and GE in the data we examined. 
Moreover, even unsophisticated examination of the data by the 

1^ court revealed glaring discrepenc ies in the data: 

the test results for some of the students were plainly 
bizarre. For example defendent's Exhibit 52 shows CAT 
scores for 197B Glennville seniors for the tenth 
eleventh and twelfth grade administrations of the test 

§ and the twelfth grade reexamination where necessary. 

Mickey A.'s reading score on the CAT was 3.8 CBE units].. 
One year later the score was 8.0. At the time of his 
senior retest, his score had regressed to ^.1. Renwick 
F. wer from a ^.3 in reading and 3.9 in math in the 
eleven grade to 10.0 and 9.8 less than two years 

P later. Darlene J., according to Defendents' Exhibit 

53 which gives the respective figures for Reidsville, 
jumped from 3.3 in reading and 5.8 in math in the tenth 
grade to scores of 9.9 and 9.S in the twelfth. Kennie 
K.'s scores improved from S,8 in reading and ^.6 in 
math in the tenth grade to 9.0 and 9,5 on the twelfth 

H grade retest. Vernie A. at Reidsville obtained the 

following reading scores: ^.9, ninth grade; S.5 tenth 
grade; and a whopping 11.5 in the eleventh grade. 
Progress was not steady [emphasis added]. 
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According to Defendents^ Exhibit IS showing the gains by 
year for 19B0 Reidsville seniors in the math section* 
Talmadge F. showed a gain of 5.0 in his freshman year's 
CAT score. His score deteriorated each year thereafter. 
The corresponding exhibit for Glennville» Exhibit 
shows Ricky B.» after having regressed .1 in the ninth 
grade* showed an incredible gain of more than 7 grade 
levels in his tenth grade math score. <520 F.Supp. at 
492) . 

In our own examination and analysis of the data we encountered 
additional discrepenc ies and confusions. Of additional note in 
the court's statement above is the examination of individual 
student histories and the emphasis on student progress <in GE 
metric ) . 

Metric for achievement. The presentation to the court and 
the court's decision was entirely in terms of GE scores on the 
achievement tests. However our Table 14 shows that the choice of 
the metric for achievement (as discussed in section 1.3.1) can 
have a large effect on the conclusions about student progress 
that may be drawn from the data. In Table 14 the quantities of 
interest are estimates of the true correlation between student 
progress and remedial/regular curricula exposure* a key issue in 
the plaintiff's presentation of evidence. Thus the issue of a 
metric for achievement has considerable practical import because 
of possible effects on the results of the analysis. 

Key questions about growth . Although there are 
considerable impediments to the analysis of student progress 
that arise in the reanalysis» questions about student progress 
remain highly pertinent. 
For exe^mple: 

1. How well or poorly are students progressing in the lower 
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tracks? This questions involves the descriptive analysis of 
student rates of progress. 

S. Uould students in the lower tracks have made better progress 
in regular rather than remedial courses? This question involves 
the relation beti^Meen curriculum and student progress and also 
the effects of ability grouping . <cf . 540 F.Supp. 761 at 764). 
3. Was the failure of many students to meet the graduation 
requirements <9.0 GE) largely a result of educational deficits 
built up before the ability grouping was implemented? I.e.y was 
their failure unaffected by the district's policies? This 
question involves the correlation of change and initial status 
and possible adjustment for intitial differences in status. 
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LISTING OF SYSTEM FILE INFORMATION 

FILE GAREC (CREATION DATE s 09/0V8t ) 

DOCUHEMTATION FOR SPSS FILE 'GAREC • 

LIST OF THE t SUOFILES COttPRISING THE FILE 

GAREC Ns 50 

DOCUHEMTATION FOR ItlE 66 VARIABLES IN THE FILE 'GAREC • 



REL VARIABLE VARIABLE LAB'L 
PC" NAME 



HISSING PRT 
VALUES FMT 



06/2 t/0S 



PAGE 



1 


SEQNUM 




NONE 


0 


z 


SUBFILE 




NONE 


A' 


3 


CASUST 




NONE 


4 


4 


IDNUM 


STUDENT IDENTIFICATION NO 


999 • 


0 


5 


ELEHREAD 


GR 4-6 GPA 


999* 


0 


6 


ELEHLA^IG 




999* 


0 


7 


cLEHtUTH 


GR 4-6 GPA 


999. 


0 


s 






QQQ 
TTT» 


A 
V 


9 


lOMAHATH 




999. 


0 


10 


GPAEN6 




999. 


0 


n 


UE»)G 




999. 


0 


(2 


CENG 




999. 


0 


13 


GPAMATH 




999. 


0 




UHATH 




999. 


0 


15 


CMATH 




•999. 


0 


16 


GPADREAD 


DEV READING COURSES 


999. 


0 


♦7 


UDREAO 




999. 


0 


IS 


COREAD 




999. 


0 



4 ■■ 
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LXST»46 OP SYSTEH PILE XNFORHATXON 



DOCUMENTATION FOR THE 66 VARIABLES IN THE FILE '6AREC * 

REL VARIABLE VARIABLE LABEL NISSIW PRT 

POS NAHE VALUES FMT 



19 


6PADHATH 


DEV HATH COURSES 




999. 


0 


20 


UDHATH 






999. 


0 


21 


CDHATH 






999. 


0 


22 


6PARREAD 


REMEDIAL READING 


COURSES 


99 V\ 


0 


23 


U7READ 






999. 


0 


24 


CRREAO 






999. 


0 


25 


GPARHATH 


REMEDIAL HATH COURSES 




999. ^ 


0 


26 


URriATH 






999. 


0 


27 


CRMATH 






999. 


0 


20 


TOTCRED 






999. 


0 


29 


riCTREAD 


U PASCt 
0. FAIL 




999. 


0 


30 


HCTNA7H 


t. PASS 
0« FAIL 




999. 


0 


31 


SATVERB 






999. 


0 


32 


SATKATH 






999. 


0 


33 


RACE 


0* HHITE 
1, BLACK 




999. 


0 


34 


READ4 


READING GRADE EQUIV QUARTILE 


HIDPOINT 




0 


35 


SEX 






999. 


0 



0. HALE 

1. FEMALE 

lb/ 
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06/21/05 PAGE 3 



LISTING OF SYSTEH FILE INFORMATION 



I 

OOCUIIENTATIttI FOR THE 66 VARIABLES IN THE FILE 'GAREC * 



POS 


vMlf AMPLE 

NAME 




HISSING PRT 
VALUES FHT 








999. 


0 


17 
Ow 


MATII7A 
iiA in f o 


HATH ADAnB BkMlTilAICUT AT TA. 


999. 


0 




If CMU #0 


KCAIIANw ImAUc cMUlVALtini AT 7o 


999. 


0 




MATM77 




999. 


0 


40 


DFAn77 




999. 


0 


61 
f 1 


MATII7A 




999. 


0 


■f c 


DFAn7A 




999. 


Q 


6t 






999. 


0 


64 
■ft 


RE An 70 




999. 


0 




REAIIPATT 


FUAI T<UI rnilD*iF UATTFDM UTI f«ll 


. 999. 


0 


46 


MATUPATT 


HATMFMATTrA rmiOQF DATTFDM XJn t.lQ 
nilincilMIA^9 UUUKdC rAllCKFI llU« I"*1t 


999. 


0 


67 


MAIIIMARK 




999. 


0 




READMARK 




999. 


0 


69 






999. 


0 




riwrc Tm C Mil 




999. 


0 


51 


RHATII 




999. 


0 


52 


VARHATH 




999. 


0 


53 


RREAD 




999. 


0 


54 


VARRbAD 




999. 


0 


55 


XNTHATH 




999. 


0 


56 


SIOMATH 




999. 


0 
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06/2 t ^45 PAGE 4 



I 




LISTING OF SYSTEM FILE INFORHATZON 

DOCUIEIITATION FOR THE 66 VARIABLES IN THE FILE 'OAREC • 



REL VARIABLE VARIABLE LABEL 
POS NAHE 



niSSIHG PRT 
VALUES FMT 



57 


IMTREAD 




999. 


0 


50 


SLOREAD 




999, 


0 


59 


RGREAD 


RELATIVE GROMTH IN READING 


999. 


0 




RCIMIII 


krlaiivc orukiii in nAIH 


ooo 

TTT. 


A 
V 


61 


OREAD 


ENTRY QUADRANT IN READING 


999. 


0 


62 


QMAIH 


EHTRY QUADRANT IN HATH 


999. 


0 


63 


ORQSENG 


GROSS CURRICULUH INDEX EI^LSSII 


999. 


0 


6<V 


FIIIEEN6 


FINE CURRICULUH INDEX ENGLISH 


999. 


0 


65 


GROSNATH 


GROSS CURRICULUM INDEX HATH 


999. 


0 


66 


FIIIEHATH 


FINE CURRICULUH INDEX HATH 


<>99. 


0 
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3.2 CRITIQUE AND REANALYSIS: Scheelhaase v. Woodbury 

This case concerns the non-renewal of an Iowa seventh grade 
English teacher's contract. The general Issues of this case were 
described in Section 2.1, "Evaluation of Teacher Performance". 
Low test scores of the teacher's students on the ITBS and ITED 
were reasons the school board gave for her dismissal. The 
plaintiff prevailed in the district court: "...professional 
incompetence as indicated by low scholastic accomplishment of 
students on specified tests was arbitrary and capricious, since 
teacher's professiisnal competency could not be determined solely 
on the basis of students* achievement on the tests, especially 
where the students maintained normal educational growth rates." 
(349 F.Supp. at 988-9) Although the decision was reversed in the 
Appeals Court to favor the School District, both courts seemed to 
accept the standard of normal educational progress as important 
evidence. 

What exactly does "normal educational progress" mean in this 
instance? Scheelhaase ' s expert witness, Norman Ashby, implicitly 
defined this notion by the calculations he presented to the 
Court. The evidence submitted to the Court and his testimony are 
found in the Joint Appendix for Civil Case No 73-1067 (U.S. Court 
of Appeals, Eighth Circuit). This critique examines the data and 
analysis presented by Ashby and presents alternative statistical 
methods and their results. 
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3.2.1 Background and Avallabla Data. 

In addition to the published summary of tha District and 
Appeals Courts decisions, we obtained documents from the United 
States District Court of Appeals for the Eighth Circuit in Iowa. 
These documents included the Joint Appendix (No. 73-1067) and 
th6 following briefs: Brief of the ,I«»wa Association of School 
Boards as Amicus Curiae (filed 4/5/73) ; Brief of the NEA as 
Amicus Curiae (filed 6/4/73); Appellants* Brief and Argument 
(filed 4/9/73); and the Appellants* Reply Brief and Argument 
(filed 6/20/73) . The Joint Appendix contains the available test 
data and summarizes Ashby's testimony. 

All students Grades 3-8 in the Woodbury School District take 
the Iowa Test of Basic Skills (ITBS) annually. The test is 
administered each year in January, and scores are reported to the 
schools in percentile rank and grade equivalent (6E) scales. 
Data Structure. 

The data available to us were found in Exhibit II of the 
Joint Appendix. There are four subscales for the ITBS for which 
6Es were reported: Reading (R) , Total Language (TL) , Total Work 
Study Skills (TW) , Total Arithmetic (TA) , ancl also Total 
Composite Score (C) . These scores were available in the Joint 
Appendix for an average over all students iw each of the grades 
(3-8). A cohort is defined by the students* anticipated year of 
high school graduation. Data on four cohorts were available: 
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CQHPRT 



YEAR IN 8E\1!NTH GRADE 



1972 
1973 
1974 
1975 



1966- 67 

1967- 68 

1968- 69 

1969- 70 



In evaluating the perfomance of an English teacher such as 
Scheelhaase, the Reading and Total Language scores ar«i relevant. 
Thus the basic data can be represented for Reading scores as R7Xj^ 
where cohort is represented by 7X « 72,73,74,75 and grade level 
by 1«3,4,5,6,7,8. Thus, reading score for the 1972 cohort in 
grade 4 is denoted R724. The notation for total language is 
identical (with the substitution of TL for R) . Recall that all 
scores were available as grade-equivalents (GE) . 

An average rate of growth for each cohort in each skill area 
(R,TL) was calculated as the difference in GE scores from third 
grade to eighth grade divided by 5s that is, for reading, the 
calculation was: (R7X8-R7X3)/5 « GR7X for 7X « 72,73,74,75. 

Then, within each skill area, each cohort's growth rates 
(e.g., 6R7X) were averaged producing an overall growth average 
for each skill area, denoted for reading by 6R. This overall 
average includes data for all the students in the school during 
the years 1966-1970, and was used as the index of normal 
educational growth. 

To compare this index with student performance ascribable to 
Scheelhaase * s Instruction, Ashby calculated Scheelhaase ' s 
students* averages in two batches. Since the ITBS was 
administered in January, the 'Grade 7» administration reflects 
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Scheelhaase's influence for the eix nonths immediately preceding, 
and the 'Grade 8' administration reflects her influence on the 
first half of the school year. Thus the two growth increments 
relevant to Scheelhaase's instruction were (for reading) RTXy- 
R7X6 and R7X8 - R7X7. Each of these were then averaged over 
cohorts . 

comparisons were made between scheelhaase's students' growth 
averages and the overall growth averages within her content area 
of reading and total language. Because the averages seemed 
comparable (i. e., they were not very disparate), Ashby asserted 
and the court agreed that Scheelhaase's students made normal 
educational progress. Table 1 summarizes the indices Ashby 
presented. 



Insert Table 1 here 



TABLE 1 



INDICES FROM SCHEELHAASE TESTIMONY 



SCHEELHAASE INCREMENTS 



COHORT 


GR7X 


GTL7X 


R6-7 


R7-8 


TL6-7 


TL7-8 


72 


.90 


.86 


.80 


.90 


.60 


1.0 


73 


.96 


.86 


1.1 


.70 


.80 


.70 


74 


.86 


.82 


.90 


.90 


.60 


1.0 


75 


.92 


.86 


1.1 


.80 


.80 


.90 


AVE 


.91 


.85 


.98 


.83 


.70 


.90 


Compare 


.91 VS 1/2 


(.98 + 


.83) 









.85 VS 1/2 (.70 + .90) 



1 r'r-i 



3.2.2 Reanalysls of 6E Data 

The available! data permit a longitudinal analysis of the 
progress of each cohort (72,73,74,75) in each of the relevant 
skill areas (i.e., R, TL) from third to eighth grade in the 
metric of grade equivalent scores (6E) . The most appropriate 
initial examination of the data is to plot thu 6E scores against 
grade. Figure 1 contains these eight plots (4 cohorts and 2 
subtests). Examination of the plots indicates that, at least for 
the average over students, the GE metric of constant rate of 
growth is valid. Fitting a straight-line growth curve to each of 
the plots yields results shown in Table 2. 



Insert Figure 1 and Table 2 Here 



Table 2 clearly shows a strong straight-line tendency in the 
GE scores. The sc[uared multiple correlations for these fits all 
exceed .97. The slope of each straight-line fit represents the 
rate of growth for each cohort on each subtest over the span 
grades 3-8. This slope is an improvement as a growth index over 
the index used by Ashby in that the slope utilizes all the data 
(via the least-sguares fit) instead of just utilizing the 
endpoints (grades 3 and 8) . A comparison of Tables 2 and 1 does 
not show marked differences between the slope and Ashby 's index, 
mainly because of the strong conformity of these data to a 
straight- line. 
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R72 - * 
7.5+ - * 



&.0+ 



4.5+ 
- * 



i!.y+ 

+ + _+ + +GFiflDE 

3,0 4.0 5.0 6.0 7.0 8.0 



TL?: 



7. 5+ 



6.0+ 



4.5+ 
- * 



* 



+ _+ + + — _ +„ +CiFiRDE 

3 0 4 0 5.0 6.0 7.6 8.0 



Figure 1. Plot of GE scores against grade level for Reading (R) and 
Total Language (TL) subtests for each of the four cohorts. 
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7.5+ 



6.0+ 



4.5+ 



- ♦ 
3.0+ 

+- 

3.0 



7.5+ 

TL75 

6.0+ 



4.5+ 



- * 

3 . 0+ 

+- 

3.0 



* 



♦ 



He 



+ + + + +GFiRDE 

4.0 5.0 &.0 7.0 8.0 



+ + + +GRflDE 

4.0 5.0 6.0 7.0 8.0 



^ngure 1, page 2. ^ 
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7.5+ 

R74 

6.6+ 



4.5+ * 
- ♦ 



3.0+ 

+ + ^ + ^ +GRfiDE 

3.0 4.0 5 0 6.0 7.0 8.0 



7.5+ 

TL74 

6 . 6+ 



4 5+ 

3.0+ 



+ + .+ + + +GF;flDE 

3.0 4.0 5.0 6.0 7.0 S.0 



Figure 1, page 3. 
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7.5+ 



R75 



I 6.0+ 



4 5+ 



- * 



-J 



.5+ 



TL75 



6.0+ 



4 5+ 



* 



* 



,5.0+ 

+ + + + + +GRflDE 

3.0 4.0 5.0 6.0 7.0 8.0 



- * 
3.0+ 

+ + + + + +GRRDE 

3.0 4.0 5.0 6.0 7.0 e.0 



Figure 1, page 4. 
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TABLE 2 



STRAIGHT-LINE FITS TO GRADE 3-8 DATA 



R72 

R73 

R74 

R75 

TL72 

TL73 

TL74 

TL75 



SLOPE 
.90 
.97 
.86 
.93 
v82 
.82 
.77 
.81 



95% CI 

(.88, .92) 

(.90, 1.04) 

(.83, .89) 

(.88, .98) 

(.74, .90) 

(.70, .94) 

(.67, .87) 

(.72, .90) 



RSQ 

.999 

.995 

.999 

.997 

.991 

.979 

.985 

.987 
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An important issue is a quantitative assessnent of any 
discrepancy between a neasur€< of •'normal educational growth" and 
a measure of student progress attributable (in some manner) to 
Scheelhaase • s instruction. In Ashby's testimony, such an 
assessment is handled highly informally: The comparison of Table 
1 summary indices (.85 versus .80) is carried out with no 
statistical basis. The synopsis of the testimony reports that 
"roughly it (Scheelhaase* increment) appears to be fairly close 
to average" (p. 75 Joint Appendix) . 

The use of a regression estimate for the rate of improvement 
provides a natural way to assess a sizeable discrepancy. The 
95% confidence interval for the slope in Table 2 defines an 
interval of plausible values for the rate of improvement. A 
natural comparison is whether the Scheelhaase relevant increments 
lie within tl*e cciifidence interval for the slope. 

Another problem with Ashby's growth index is that the data 
used to define "normal educational growth" included the data used 
to assess the student progress relevant to Scheelhaase ' s 
instruction. This overlap will certainly bias the analysis 
towards a conclusion that Scheelhaase ' s students do accomplish 
normal educational growth. An improved analysis would separate 
the data used to represent Scheelhaase ' s instruction. Such an 
analysis could use the four data points from grades 3-6 to 
establish a regression of GE on grade and then determine whether 
the GE scores from grades 7 or 8 fall within a 95% prediction 
interval based on this line. Table 3 reports these prediction 
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Intervals for each cohort and each subtest along with the 
observed GE scores. In each instance the GE score falls within 
its prediction interval. Note that some of the prediction 
intervals are rather wide. Thus this analysis cannot detect a 
deviation of the Grade 7 and 8 scores from the trend established 
in the lower grades. 



Insert Table 3 Here 
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TABLE 3 





PREDICTION 


INTERVALS 






m 


GRADE 7 




GRAu£ 8 






PREDICTION INTERVAL OBSERVED DATA 




UoSERVSD DATA 


172 


(7.34,7.86) 


7.5 


/D Q OR\ 

(o . 2X, o . 69; 




• 


(6.70,8.31) 


7.5 


V / . 34 , 9 • 5U) 






(6.61.7.79) 


7.2 


( / .J«l,o.77J 


8.1 


\ / D 


(6.10,8.10) 


7.2 


(6.82,9.24) 


8.0 


_''L72 
• 


(6.64,8.56) 


7.2 


(7.33,9.67) 


8.2 


:L73 


(5.01,9.39) 


7.0 


(5.43,10.75) 


7.7 


rL74 


(5.74,8.66) 


6.8 


(6.26,9.82) 


7.8 


:L75 


(5.23,8.77) 


6.8 


(5.70,10.00) 


7.7 
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3.2.3 Examination of Alternative Functional Forms 

Although grade-equivalent Bcores are constructed with the 
Intent of yielding a constant rate-of -Increase (I.e., 1 6E unit 
per grade) , there are no guarantees that student scores will 
exhibit a constant-rate-of -change (i.e. straight-line growth). 
We used the Grade 3-8 data to examine the alternative functional 
forms for student progress, one common alternative to a 
constant-rate-of-change is a growth rate that depends on the 
student's current status — a constant relative rate of change 
conforms to exponential growth towards ar. asymptote. Table 4 
presents both relative rate of change indices and the actual 
amount of change. The quantities in Table 4 are defined as 
follows (where is the 6E score for grade 1) : 

^1 ■ ^1+1 " change between grade 1+1 and 1. 

^1 " °i/^i Relative rate of change. 

EXi « InXi+i - InXi Rate of change in natural log units. 

The latter two indices will be constant for exponential 
growth; the rate of change depends on status or distance to 
ceiling. Clearly, the data in Table 4 most closely conform to a 
constant-rate-of-change (straight-line growth) . 



Insert Table 4 here 
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TABLE 4 

RATE OF CHANGE AND RELATIVE RATE OF CHANGE 

Di Si EXi 

COHORT 
1972 



1973 



1974 



1975 



tADE 


R 


TL 


R 


TL 


R 


TL 


3 


0.90 


1.10 


0.23 


0.28 


0.21 


0.25 


4 


0.90 


0.90 


0.19 


0.18 


0.17 


0.17 


5 


1. 00 


0.70 


0.18 


0.12 


0.15 


0.11 


6 


0.80 


0.60 


0.12 


0.09 


0.11 


0.08 


7 


0.90 


1.00 


0.12 


0.14 


0.11 


0.13 


3 


1.00 


1.5 


0.29 


0.44 


0.26 


0.36 


4 


1.20 


0.5 


0.27 


0.10 


0.24 


0.10 


5 


0.80 


0.8 


0.14 


0.15 


0.13 


0.14 


6 


1.10 


0.8 


0.17 


0.13 


0.16 


0.12 


7 


0.70 


0.7 


0.09 


0.10 


0.09 


0.10 


3 


0.80 


1.10 


0.21 


0.30 


0.19 


0.26 


4 


1.00 


0.90 


0.22 


0.19 


0.20 


0.17 


5 


0.70 


0.50 


0.13 


0.09 


0.12 


0.08 


6 


0.90 


0.60 


0.14 


0.10 


0.13 


0.09 


7 


0.90 


1.00 


0.13 


0.15 


0.12 


0.14 


3 


0.80 


1.30 


0.24 


0.38 


0.21 


0.32 


4 


1.20 


0.70 


0.29 


0.15 


0.25 


0.14 


5 


0.70 


0.60 


0.13 


0.11 


0.12 


0.11 


6 


1.10 


0.80 


0.18 


0.13 


0.17 


0.13 


7 V 


0.80 


0.90 


0.11 


0.13 


0.11 


0.12 
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Percentile Rank Data. The test results were also reported In 
"percentile rank" form. Percentile ranks are not in general a 
useful metric for assessing change over time. In addition these 
particular data were extremely volatile and their accuracy is 
unclear. 

3.2.4. Shortcomings of th^ Data for Evaluating Teacher 
Performance . 

1. The January testing date further obscures the attribution of 
student test performance to a particular teacher's instruction. 

2. The lack of a comparison group (e.g. from similar districts) 
for progress in seventh - eighth grade instruction makes 
assessment of "normal educational progress" more difficult. 

3. Group (school-averaged) data is less useful than individual 
data for assessing student progress. 

4. Grade-eq'jiivalent metric may :iot be best (or even adequate) 
for analyzing student progress. 
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SECTION ^ 
CONCLUSIONS AND RECOMMENDATIONS 



The purpose of this section is to review some of the 
lessons learned from this research and to indicate possible 
future directions for methodological research* Section ^.1 
restates some of the broad principles and procedures for the 
design and analysis of investigations of student progress. 
Section ^.2 sketches some unsolved methodological problems 
arising from some of the court cases examined. 



4.1 IMPROVED DATA COLLECTION AND ANALYSIS 

The previous sections of this report have demonstrated two 
conclusions; (1) questions about student progress play a key 
role in many court cases* and <a) current practice in 
measurement of achievement* data collection and management, and 
statistical analysis have serious deficiencies. 

4.1.1 Data Collection and Management 

School districts regularly assess students using group- 
administered achievement tests. Such testing represents a large 
investment in money and time for the schools, for 
administrators, for teachers and for students. Yet, local school 
agencies make relatively little use of the test data which they 
accumulate. In particular, test results are presented in a way 
that describes only the current status of students; the data are 
presented as a static "snapshot" of achievement without any link 
to prior levels of performance. Even the management of test data 
reflect these limitations. Whether the test results be stored 
as hard copy or electronically, the achievement data are 
typically organized as separate yearly files, which may be 
located on separate physical devices and even in separate 
geographical locations. 

A key to the improved use of achievement test data is to use 
performance on repeated tests to describe student learning. A 
student's score at a single point in time cannot be used to 
measure learning; collecting together scores from previous 
testings is necessary for the analysis of student progress. A 



student's "cumulative folder'* is organized in this manneri but 

^ these are rarely stored electronically nor uniformly maintained. 

Design of data collection. The key. design questions 
involve what data to collect and when (or how often). Thus (in 

p addition to the typical question! How many individuals? which is 

often predetermined by the number of students and schools 
involved in the court case) design decisions include (i) the 

§ type and content of the achievement measure? (ii) the type of 

test score used to represent achievement; (iii) the number of 
and time between the assessments of achievement; and (iv) the 

§ type of exogenous medsure(s) (e.g. curriculum^ school program^ 

initial status^ demographic identification) to be linked (e.g.y 
correlated) with individual or group progress. Standardized 

^ achievement tests were the predominant form of test instrument 

used in the court cases^ in large part because these are the 
most avaliable measures. In many of the cases (e.g. Tattnall 

H County) the court gave considerable attention and scrutiny to 

the appropriateness and relevance of these testsy particularly 
th^e match of the test and the content of the curriculum 

« 

p (curriculum validity following Debra P. v. Turlington) . 

Clearlyy an alternative to standardized achievement tests are 

tests locally produced^ which can be tailored to the curricula 
I and extended to content areas for which standardized instruments 

may not be available. 

The type of achievement measures m^^intained in the student 
I individual history is extremely consequential. Of the types of 

achievement scores reviewed in section 1.3.1 the scale score is 
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the most appropriate measure with which to analyze student 
progress. The grade equivalent metric is the predominant ^ype 
of score used> with advantages of simplicity of interpretation 
and implicit metric of expected progress* Raw scores also have 
some suitability for charting progress but only within the same 
level of the test. Particularly unsuitable for the analysis of 
student progress are percentile r-^nk score and normal curve 
equivalents (NCE) • 

The number and spacing of the longitudinal observations is 
the most visible design decision <or constraint). Two 
observat ionsy spaced a year apart constitute the most common 
design seen in the court cases. (Analyses^ such as gain scores 
and regression methods^ for the two-observation designs are 
descibed in Section 1.3.3.) As Rogosa et al. <198e> p. 7^^) 
state: "Two waves Cobservat ions] are better than one> but maybe 
not much better. Two data points provide meager information on 
individual change^ and thus the measurement of change often will 
require more than the traditional pre-post data." Rogosa and 
Willett (19B5> section 1.3) detail the shortcomings of two-wave 
data for the correlation of exogeneous variables (e.g.* race or 
curriculum) with individual change. The statistical methods 
based on growth curve analysis require more than two 
observations, even for fitting the simplest p straight-line 
growth curve. The timing of the observations is also an 
important cesign descision. Measures collected a year apart 
predominate in the court cases because of traditional school 
testing practices. Denser observation schedules (e.g. 
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bimonthly) would make the collection of multiple (more than two) 
observations on each individual far more prsictical and relevant. 
Also the time of the school year at which the achievement 
measures are obtained can be important; for example^ the annual 
January testing from Scheelhaase v. Moodburv (section 3.2) 
caused considerable difficulty in any assessment of a single 
teacher *s performance (Sept ember --June) .of teacher performance. 

Often* a main purpose of the longitudinal analyses is to 
investigate a question about possible correlates of change. 
That is» the correlation between an exogenous variable and 
student rate of progress is of central interest. Whether race is 
related to rate of learning is a centra) question in the racial 
discrimination or unequal educational opportunity suits in 
section 5.2 <e.g., Serna v. Portajes t section 2.2.2). The most 
common exogeneous variables in the court cases were curriculum 
or school program variables (In which curriculum or program to 
student make the most progress?) » demographic or racial 
identification variables (Are minority student making the same 
progress as majority students?), or measures of initial status 
(Are the students with better initial standing making the most 
progress?). As seen in the Tattnall County reanalysis (section 
3.2) the quality and choice of the exogenous variable is 
consequent i al . 

A. 1.2 Statistical Analysis 

Statistical model for individual growth . Psycho logical 



learning theory and biological growth research provide a variety 
of complex models of individual growthf such as polynomial 
growth curves* logistic growth curves and simplex models. The 
simplest model » and the one used throughout this monograph* is 
the straight-line growth model* 



( (t) « f (0^ + « t 
P P P 



where fp^*^ is the true score of person p at time t * 1* S* 

• ••9 T and is the constant rate of change for person p« 

Thus* estiinates of provide a simple index for individual 

rate of learning. The parameter* is closely related to the 

amount of true change; for example* in two-wave (or pre-post) 

data* true change defined as i (tg.) - ( (ti) is equal to 

P P 

«p(ta-ti). In addition* the estimation of true scores ( (t) 

from observed scores X (t) provides information about level as 

P 

well as rate of learning. 

The straight-line growth model is useful for heuristic 
reasons because of its simplicity* as it yields a simple index 
for individual rate of progress. In addition* Rogosa and Wilie'jt 
(1985) point out that* **in applications* straight-line growth 
serves as a useful approximation to actual growth processes" <p. 
S05). Moreover* when observations at only a few time-points are 
available* as with the Tattnall County data in section 3.1 where 
T = ^* the data may only justify the estimation of a constant 
rate of change. Although many uses ot straight-line growth 
curves can be justified* non-linear growth functions may be 
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crucially important in many appl icat ionsy as in detecting 

^ effects of programs or teachers and in assessing deviations from 

'^normal educational progress^** Clear ly^ this is a v^orthwhile 
area for future statistical research and application; the tools 

9 and applications presented in this report serve at least as a 

useful first step. 

Descriptive analyses of growth rates. The straight-line 

0 growth model does allow one to fruitfully compare individual 

learning for different individuals* For example^ consider the 
four examples in Figure 1 of section The figure shows the 

# estimated individual learning ^^curves'* obtained for four 

students by fitting a straight line to each student's individual 
longitudinal data obtained in the high school grades, Student 

% <a) exhibits a rapid increac^e in reading achievement from a 

relatively low initial level of achievement. Student (b) starts 
out somewhat higher than student (a) and grows at a similar 

% rate. Note that the observed scores are closely approximated by 

the fitted line. Student <c) exhibits slow growth from a 
relatively low initial achievement levels but the observed 

^ scores are more erratic than those for students <a) and <b). 

Finally^ student (d) shows a high level of achievements but no 
growth due to the ceiling effect of the tesrt used, 

d When describing the learning of a group of individuals, 

the distribution, over individuals^ of empirical rates of 
learning is informative. The five-number summary of empirical 

% rates such as in Table 3 of section 3,1 is one useful way to 

describe both typical rates of learning and the degree of 
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variability in rates of growth among individuals. Also of 

interest is the variability in b , or estimates of « . Thus 

P P 

2 

a^^ IS a key quantity for investigation. Similarlyt we may want 

to describe the variability in level of performance at each 

g 

time, (^<t) . As it turns out, <r.... has a functional 
P C <t ) 

dependence on time, and investigation of the form of this 

function leads naturally to the definition of a ''center ing** 

point and a scaling factor associated with the time scale. 

These have been denoted ^nd k , respectively. (See F^ogosa 

and Willett, 1985.) Be «h and are properties of the 

particular collection of straight-line growth curves. 

Correlation of change and initial status . Another quantity 

of central importance is the correlation between change, b, and 

initial status, ((tj), where tj indicates initial time of 

measurement. As discussed in Rogosa and Willett (1985), the 

choice of t. is of critical importance because P.... is 
A C ( t )b 

functionally dependent on time. (The definitions of t*^ and % 
also arise naturally from an investigation of this dependence? 
see Rogosa and Willett, 1985.) Our statistical procedures 
provide a maximum likelihood estimate of the correlation between 
true rate of change and true initial status; the correlation 
between observed change and observed initial status is well- 
known to have a strong negative bias (see Rogosa et al* 1982). 
The correlation is used to investigate whether those with lowest 
initial status make the most progress (negative value) or those 
with the highest initial status make the most progress (positive 
value) • 
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Correlation of exogenous variables with growth. More 
gen'srallyf there is interest iv ways of describing systematic 
individual differences in growth^ as indicated by the quantity 

where W is some exogeneous background character istic ^ for 
example^ a characteristic of the school curriculum. The 
question addressed is w\ether students who experience certain 
values of U tend to exhibit more or less growth than students 
who experience other values of M . Our statistical procedures 
prov^de maximum likelihood estimates of this correlation? as in 
the correlations of progress with curricular ividices in section 
3. 1. 

In investigating systematic individual differences in 
growth? it is of course important to have a model for individual 
differences in growth. Rogosa and Millett (1985) state 
"Individual differences in growth exist when different 
individuals have different values of . Systematic 

individual differences in growth exist when individual 
differences in. a growth parameter such as can be linked 

with one or more U's . ** (p. SOS) Ue use the simple 
representation 



i U BW W 



Thus non-zero values of fi^^ indicate that W is a predictor 
of growth. Alternatively? p^^^ is a useful summary quantity. 

The typical procedure is to correlate the value of the 
background demographic variable or curricular variable with 
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performance at a given time. That is> the cross sectional 
# correlation is computedf sometimes for every occasion in time. 

For example^ with a background variable! W f correlations of the 

test score with U at grade 99 IO9 11 9 IS would be computedy and 
9 from these correlations conclusions about learning are 

attempted . 

Rogosa and Willett <19B5) have shown that such cross-- 

H sectional correlations cannot inform about student progress. To 

illustrate9 consider a situation where the correlation between 
true rate of change and the background variable is zero. Then 

t the correlation between the true test scorey C<t)9 and the 

demographic variable9 Wf at any one slice in time could be big 
or small. Consequent ly 9 ^£(t)W ' ^^e^HV doesn't inform about 

§ systematic induvic'ual differences in learning. The reverse is 

true &lso. Consider a ^^'emographic variable for which p^^^^ is 
large. Regardless9 the correlation between the background 

d variable and a test score at a specific time can be positive9 

2ero9 or negative depending upon the time chosen for the cross- 
sectional correlation. Obviouslyy no useful conclusions about 

i learning can be drawn from the cross-sectional correlations. 

Consistency of individual differences. The index Y was 
proposed by Foulkes and Davis <19B1) as an index of trackingy 

9 and is defined as the probability that two randomly chosen 

growth curves do not intersect. High values of Y indicate 
high consistency of individual differences over time. Another 

§ way of interpreting Y is to note that high values of Y 

denote "the maintenance over time of relative ranking within the 
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response distribution" <Foulkes & Davis> 1981 i p. ^39). Thus y 

indicates the stability of individual differences. If a 

collection of individual growth curves have a high value of )S 

that indicates that individuals that started out relatively high 

maintain that advantage and individuals starting out lov^ retain 

that disadvantage (regardless of the overall growth rate). 

Individuals with a low value of y are those whose relative 

P 

standing changes considerably over the time period. 
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^.2 Further Methodological Problems 

In the application of statistical analyses of student 
progress to the issues raised in the various court cases» a 
number of unsolved or unrecognized methodological problems 
became apparent. The discussion in this section may serve to 
direct some future technical research and development or help 
those involved in future court cases to recognize the 
limitations of empirical evidence or research capabilities. 

^.S.l. Assessment of Normal Educational Progress 

The determination of normal or adequate educational progress 
and the assessment of deviations from such are dominant 
methodological issues in many of the court cases reviewed in 
this project. Clearly^ such determinations require ^substantial 
longitudinal data on student achievement and serious statistical 
analyses. In section ^.1 some general recommendations for the 
design and analysis of longitudinal data on student achievement 
are presented. In section 1.3.S» particularly section 1.3.S.39 
previously used methods for determinations of adequate progress 
are reviewed^ and the reanalyses in Section 3» particularly 
section 3.S» present and illustrate relevant statistical methods 
and problems. 

Three basic questions for which neither answers ur even 
complete methodological approaches are available are: 



1. What is the functional form of normal educational 



progress? How does the functional form differ for 
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minority groups^ different socio-economic levelSf etc.? 

S. In what test-score metric (raw scores GE> scale score) 
should expected status and progress in achievement be 
reported? Can these be linked to progress in curricula 
(e.g. basal reading sequence)? 

3. What are useful methods for detecting and assessing 
deviations from normal educational progress (i.e.i 
effects due to teachersy curricula^ grouping practices?) 

^.S.S. Classification Into Tracks by Standardized Test Scores 

Ability grouping will necessarily rely upon imperfect 
classification schemes. Clearly, the more imperfect the 
classification mechanism the more difficult it will be for 
school districts to defend the "educational justification" for 
its ability grouping procedures. In particular, an important 
technical problem is to determine the error rates and optimal 
classification procedures when fallible measures such as 
standardized achievement tests (which contain errors of 
measurement) are used to form the ability groups. In the Dillon 
County case (section 2.4.1.3) the district was found to be in 
violation of Title VI of the Civil Rights Act as a result of its 
use of the total CTBS score for ability grouping. Particularly 
pursuasive to the judge was the testimony that bivariate 
classification — math scores used to form ability groups for math 
classes and reading scores used to form ability groups for 
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reading classes — wuuld have greatly reduced the number of 
racially identifiable classrooms. Classification into groups on 
the basis of fallible scores has a small psychometric research 
literature which is summarized below. None of these procedures 
have been considered in the litigation involving ability 
grouping and the application of these proccjdures in school 
settings has not been explored to our knowledge. 

Lord (1962) investigated cutting scores and errors of 
measurement for two selection variables <i.e. reading and math 
scores) assumed to be bivariate normal. Starting with the 
assumption that multiple cutting scores are optimum in the true 
score space, he determined the shape of the best selection 
region when fallible scores were used. Lord derived a formula 
^ for the cutting contour and illustrated the shape, direction and 
magnitude of the distortions produced by errors of measurement; 
his equation 19 expresses the form of the contours in terms of 
^ test-score statistics. Whereas the selection region defined by 
multiple cutting scores in the true score space is a quadrant 
cut of by a right angle with sides parallel to the co-ordinate 
^ axes, the best selection region based on the fallible observed 

scores is a curve resembling a hyperbola (Lord, Figure 1). Lord 
concludes that "other things being equal, the lower the 
^ reliability of the predictors, or the higher the correlation 

between them, the greater the difference between the multiple- 
cutting scores selection region and the optimum selection 
^ region" (p. 29). 

More recently, Huynh (1982) presented a Bayesian approach 
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to the determination of cutting contours when multiple test 
scores are useo in the specific classification context of 
granting or denying mastery. He also discussed the influence of 
the loss ratio on the cutting contour and the distortions due to 
errors of measurement. Huynh illustrates his methods by the 
constuction of cutting contours for bivariate scores. Huynh 
describes the effects of errors of measurement by plotting 
cutting contours for bivariate normal X. f having different 

values of a common reliability coefficient^ p (Huynh» Figure 2). 
In earlier papers Huynh (1976^ 1977) considers a Beta-Binomial 
model for classifications of examinees into two groups (mastery^ 
non-mastery > . 

Assessment of (Differential) Mobility Among Tracks 
In racial discrimination or equal educational opportunity 
cases involving ability grouping or other classifications of 
students^ issues of mobility among the tracks^ especially 
differential mobility for majority and minority students^ are 
central. Section 1.3.S.5 explores some issues and methods in 
improvement in classification status. Section 2.^.1 provides 
examples of cases involving ability grouping; differential 
mobility is illustrated in Figure 1 of section 2.^. Section 
2.^.2 discusses special education cases. Howeverf no criteria 
and little methodology exist for the systematic study of student 
mobility among classifications. For examplef the Office of 
Civil Rights standards for racial ident if iabi 1 i ty of classrooms 
(± 20 percent) has no clear application in this context. Key 
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quest i ons i nc 1 ud& : 



1. How can standards be set for mobility among tracks? 

5. How can differential mobility among groups (e.g. 
minority and majority students) be quantified? 

3. How can measures of progress in achievement be used to 
indicate reclassification of students (e.g. 9 remedial to 
regular classrooms or special education to less 
restictive environment)? 
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4 Annotated Bibliography of Additional Sources 

The work of this project drew upon three distinct literatures 
relevant to the analyses of achievenent data in educational court 
cases. These three literatures are classified according to the 
professional identities of the authors as: (1) statisticians; (2) 
legal professionals; and (3) educational researchers. The lack of 
overlap between the literatures was surprising. The annotated 
bibliography below is one small attempt to bridge the gaps among 
these areas. 

I Statisticians 

Fienberg, S.E., and Straf, M.L. (1982). Statistical assessments as 
evidence. Journal of the Roval statistica l Society. A. 1A5. part 4, 
410-421. 

Th3 authors discuss the history of statistics in the courts. 
Statistical evidence was originally considered as hearsay and not 
admitted as evidence. More recently, methodologies are questioned 
and attorneys and judges are learning about statistical methods. 
However, Pienberg and Straf note that "there are no well accepted 
standards for the preparation, organization and docu-.rtentation of 
statistical evidence" (p. 416) . When presented as evidence, 
statistics are regarded as outside the realm of s^,-j.ence and inside 
the realm of justice. This necessitates reinterpretation and 
reformulation (see p. 420) . 
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Fienberg and Straf present results on the number of cases using 

^ statistical terns (e.g. degrees of freedom) and show that there has 

been a drastic increase since 1960. They also note that a Panel on 

Statistical Assessments as Evidence in the Courts has been funded by 

^ the National Science Foundation. They discuss legal versus 

scientific inquiry (justice versus truth) and consider two cases 

(employment discrimination and microwave advertisements) . Several 

^ interesting observations are capsulized in quotations: 

*'In examining court records, statisticians might well express 
surprise as to the methods used and analyses used by statistical 
witnesses or, more important, as to the methods not used or analyses 
not done" (p. 415) . 

• "The lawyer may ask the expert to present material in a 
selective fashion or . . . seek out another statistician whose 
findings are more acceptable. This type of situation can present the 
statistician with a moral dilemma" (p. 415). 

"If the opposing sides have both introduced expert statistical 

• witnesses, the court almost certainly will be confronted with 
conflicting testimony" (p. 416) , 

^ Fienberg, s.E. and Kadane, J.B. (1383). The presentation of Bayesian 

statistical analyses in legal proceedings. The Statistician . 88- 
98. 

^ The authors discuss the use of Bayesian methods in legal cases. 

They illustrate the approach with a simple example. Then they 
discuss how some key legal phrases (e.g. "guilty beyond a reasonable 

^ doubt") may be interpreted in the Bayesian frameworho in actual case 

is re-examined in some detail. 
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Kaye, D. (1980) . Naked Statistical Evidence. Yale Law Journal . 
601-611. 

This article Is a book review of Quantitative Methods In Law: 
Studies In the Application of Mathematical Probability and 
Statistics to Legal Problems , by Flnkelsteln (1980) . It centers on 
the objective and subjective Interpretations of probability that 
occur In jury selection discrimination cases. The standards of 
burden of proof and preponderance of evidence are discussed In 
relation to showing that some event Is ^.ore likely than chance to 
occur. 

Kaye, D. (1982) . The Numbers Game: Statistical proof of 
discrimination. Michigan Law Review . 80 . 837-856. 

The article reviews Baldus and Cole (1980) , statistical Proof 
of Discrimination . Kaye highlights how statistics are used in 
establishing a prima facie case of discrimination, the use of 
hypothesis testing, p-values, prediction Intervals and Bayesian 
analyses . 

Kaye, D. (1982). Statistical Evidence of Discrimination. Journal of 
the American Statistical Association . 77, 773-783. 

Kaye discusses various statistical techniques for presenting 
Inferential Information to a court and illustrates with two jury- 
selection discrimination cases. Some time is spent discussing the 
legal meaning of discrimination in various contexts. He notes that 
standards for handling statistical evidence in courts are still 
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evolving and notes three Issues not yet fully resolved: (a) Is 
statistical evidence of under-representatlon sufficient for finding 
dlscrlnination? (b) when nay population analyses be used instead of 
statistics? (c) What methods should be used to infer that jury 
selection is independent of, say, race? He discusses each of these 
in turn. 

Statistical techniques considered Include p-values, hypothesis 
testing, prediction (or confidence) intervals, presentation of the 
likelihood function, and Bayesian analyses. He considers the 
Bayesian analysis still too controversial for presentation as 
evidence in court. 

The article is followed by comments from several authors. 

Downton, F. (1982). Legal probability and statistics. Journal of the 
Roval Sta tistical Societv-Series A . 145 (4) . 395-402. 

The author proposes a clarification of legal cases in which 
statisticians may be Involved. The taxonomy is based on the level of 
statistical sophistication and/or the nature of the role of the 
statistician in the case. The categories are summarized below. 

(I) Non-numerical statistics — Evidence is verbal and descriptive 
rather than statistical or mathematical. The statistician is 
regarded as an expert witness and some attention is given to 
establishing their credentials and experience. 

(II) Statute statistics — In these instances, the statistical 
procedures are incorporated in the law. The statistician and lawyer 
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are in a more client-like relation with the statistician being 
expected to interpret statistical terminology to non-statisticians. 

(iii) Deterministic statistics — Here the problem is essentially 
statistical but is not perceived as such by the court. An example is 
given of an Irish court that regarded actuarial life expectancy 
tables as non-statistical. The attitude of the court in these cases 
is one of concern with the facts of a particular case. Hence 
expected values or general statistical results are discounted. 

(iv) Small sample statistics — In these cases, there are no 
particular statistical problems. The presentation of statistical 
evidence plays only a supporting role in the legal system. The 
probabilistic assumptions for the analysis are a fundamental part of 
the evidence. However, it is noted that the reluctance of 
statisticians to attribute "cause" may be interpreted as 
prevarication by the court. 

(V) Applied probability — In these situations there is usually a 
small amount of data available for inference, and probabilistic 
assumptions are less clear. The author gives two examples. He 
suggests that statisticians avoid cases in this category where the 
appropriate reference set for the analysis is undefined. Arguments 
about mathematical assumptions merely obscure the basic problem of 
inadequacy of information on which calculations should be based, 
(vi) Pure Probability — In this case, probability theory is 
fundamental to the case (e.g. gambling offenses) . However, 
mathematical equivalence is not the same as legal equivalence. The 
law is concerned with actual situations and the court prefers 
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specific mathematical results rather than general abstractions 
^ (though they may have some bearing) * 

Meier, P. (1986) . Damned liars and expert witnesses. Journal of the 
American Statistical Aagociation. £1, 394. 

Meier suggests that the "needs of the court are not well 
matched with the usual practice of statistics." Sources of 
^ difficulty and suggestions for improvement are outlined. Sources of 

difficulty include: 

Domains of application of statistics in law — (Sampling, Paternity 
and fingerprints, Observational data); Inference in discrimination 
cases; Position of the expert witness; Corrupting influences. His 
suggestions relate to increased attention to ethical and moral 
standards . 

Fisher, F.M. (1986) . Statisticians, econometricians and adversary 
proceedings. Journal of the American Statistical Association. Mr 
394. 

This is another discussion of statistician's interaction with 
the court. 
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II Legal Profeaalcnals 

Baldus, D.C., and Cole, J.W. L* (1980)* Statistical Proof of 
Diacrimination . New York: McGraw-Hill. 

Baldus ia profeaaor of law at the Univeraity of Iowa; Cole is a 
conaulting statiatician. In ten chapt^ra, they diacuaa a broad range 
of atatiatical techniguea and typical usage in legal proceedinga. 
The introduction aeta the perapective by noting that the validity of 
a research method (aa judged by the court) ia assessed in terms of 
the legal theory of a case. Asseaaing the "reliability" (i.e., 
relevance, accuracy) of qualitative information aupplied by 
atatiaticiana reguirea information about other factors such as the 
aelection proceaa uaed in collecting data or the characteristics of 
the original data itself. Much of the book is devoted to 
discrimination cases. Accordingly, on page 11, the authors give a 
list of decisions within schools which are commonly subject to 
claims of discrimination. There was no focus here on achievement 
testing, growth over time or the problem of determining normal 
progress. 

Chapters 1 and 2 describe various types of court proceedings 
for disparate treatment cases, disparate impact cases, and some 
other types of cases. The nature of a "proof" and threats to the 
"reliability" of quantitative analyses are discussed. It is noted 
that "an ideal proof will focus on what the substantive law 
considers relevant and only that . . ." (p. 71). 
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Chapter 3 discusses measures of actual treatment such as: 
actual number selected, rates of selection, rates of rejection, 
rates of representation. (Chapter 2 discussed the use of "actual" 
measure, "ideal" measure, and "summary" measures [ide&l-actual 
discrepancy] ) . 

Chapter 4 covers relevant comparisons, data collection and 
analysis. Examples include applicant flow, labor force and 
definitions of general popular- ion. The remaining chapters are 
briefly summarized below. 

Ch. 5: Discussed summary measures (ratios vs. absolute 
difference); regression and correlation measures; and viewing 
distributions separately for subgroups (e.g. minority/majority). 

Ch. 6: Covers more complex models to take into account job 
qualifications. 

Ch. 7: Covers subgroup comparisons and matching groups. 

Ch. 8: Discusses multiple regression and gives an example of 
investigating a salary dispute adjusting for job qualification. 

Ch. 9: Discusses statistical inference — tests and confidence 
intervTils, sample size, the logic of inference and threats to 
"reliability." 

Ch. 10: Is devoted to the interpretation of statistical 
analyses. Three major considerations are: (1) the risk of error 
from small samples, and sampling variability; (2) nonprobabilistic 
threats to validity (e.g. design, measurement error, faulty data) ; 
and (3) inferences suggested by other relevant evidence in the case, 
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Pinkelstein, M. (1978). Quantitative Methods i n Law, studies in the 
Application of Mathematical Probability an d statistics to Legal 
Problems . New York: The Free Press. 

This book is another broad survey, nine chapters, covering 
cases such as jury discrimination, voting, economic concentration, 
solvency controls, administrative proceedings (using regression) and 
compensation for wrongful death. Chapter 1 provides an introduction, 
discussing the importance of mathematical probability to 
applications of legal uncertainty. In Chapter 7 the author notes 
that "the unfamiliarity of regression techniques has undoubtedly 
impeded their acceptance" (p. 213). An attempt is madft to describe 
and illustrate the technique. The appendix shows a legal application 
of Bayes* theorem. 

Finkelstein, M.O. (1980). The judicial reception of multiple 
regression studies in race and sex discrimination cases. Columbia 
Law Review r 80, 737-754. 

The author discusses the "statistical war" that has occurred in 
courts since 1975. He discusses discrimination cases related to 
promotion, hiring, and back pay. Among the statistical problems 
considered are inappropriate variables, qualitative factors, data 
errors and "reverse" regression. 

Fisher, F.M. (1980) . Multiple regression in legal proceedings. 
Columbia Law Review ^ 80, 702-736. 

Most of the article is devoted to simple exposition of multiple 
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regression. The author covers uses of multiple regression, the 
purpose, how It worK estlnatlon, and the assumptions underlying 
the model. Multiple independent variables and the erroneous 
inclusion or exclusion of variables is discussed. Another section 
deals with "goodness-of-flt." 

Fisher discusses three types of legal proceedings and whether 
regression is appropriate to each. He finds that multiple regression 
is appropriate for wage discrimination cases, of dubious use for 
investigating anti-trust damages in price fixing cases, and 
"dangerously misleading" when investigating punishment as a 
deterrent to crime. 

Zeisel, H. (1978). Statistics as legal evidence. In W.H. Kruskal and 
J.M. Tanur (eds.). International Encycl opedia of Statistics (1118- 
1121). New York: The Free Press. 

This article is divided into three parts. The first part 
discusses descripti ve statistics . Legal precedents are given for 
using descriptive statistics and survey estimates. Surveys are 
covered in some detail discussing how they overcome the "hearsay 
objection" (state-of-mlnd surveys are an exception) and the fact 
that surveys sometimes Involve experimental design. Surveys must be 
meticulously executed and documented in order to meet the "peculiar 
requirements of legal evidence" (i.e. double scrutiny by opposing 
counsel). Because double scrutiny is "overexacting," one flaw can 
result in discounting the whole survey as evidence. 
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Another Important point is that some issues of interest are 
only clarified during the trial. Thus, surveys must be prepared in a 
general way to anticipate these interests while introducing minimal 
bias. [Note: the same is probably true for any methodologi^ ] 

The second part of the article discusses statistical inference . 
Its uses are illustrated by anecdotes of parking violations, tax 
evasion, cheating on a teef^ , and proof of discrimination in jury 
selection (by comparing actual with expected distribution) . It is 
noted that there are problems with imputing causality from 
nonexperimental data (one must show that other plausible causes did 
not in fact cause the effect) . 

Finally, part three discusses the " evaluation of specific 
lazfifif." Here the concern is with the reliability of observation and 
testimony. The court is concerned with "evidentiary power" of, say, 
a blood test to establish paternity. 

Shoben, E.W. (1978). Differential pass-fail rates in empJ'jymert 
testing: Statistical Proof under Title VII. Harvard Law Review . 21, 
793-813. 

Cohn, R.M. (19B0a) . On the use of statistics in employment 
discrimination cases. Indiana Law Journal ^ §5, 493-513, 

Shoben, E.W. (1980) . In defense of disparate impact analysis under 
Title VII: A reply to Dr. Cohn. Indiana Law Journal , 55, 515-536. 
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Cohn R. M. (1980b) . Statistical laws and the use of statistics in 
law: A rejoinder to Professor Shoben. Indiana Lav Journal . ^5 . 537- 
549. 

These articles are all related to employment discrimination and 
the use of the four-fifths rule in Title VII. The first article, by 
Shoben (1978) recommends the abandonment of the four-fifths rule and 
recommends instead a statistical test of the difference between 
independent proportions to decide whether pass/fail rates, say, are 
different for blacks and whites. (Appendix describes the Fisher 
£xact Test.) 

Cohn (1980a) more generally discusses "three issues concerning 
the potential misuse of quantitative information in employment 
discrimination cases" (p. 493). First, the author argues against the 
use of inference to indicate adverse impact, on the grounds that he 
believes an employer's quantitative data should be regarded as 
population data. It is not a sample from some larger population of 
interest. Second, Cohn argues that inappropriate measures are 
frequently used. For example, the ratio of qualified applicants who 
are hired is of key interest but usually not directly observable. 
Thus, "disaggregated" measures of the selection process should be 
used in conjunction with "direct qualitative analyses of the 
procedures used in the process" (p. 504). Third, the author offers 
what he believes to be the proper analysis of quantitative data to 
indicate discrimination. It includes a discussion of controlling for 
intervening variables. Cohn contends there is no need for lawyers to 
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become statisticians but they must be able to evaluate "qualitative 
information as probative" , to employment discrimination. 

Shoben (1980) strongly believes that Cohn has addressed the 
wrong legal questions. First, she notes "in class actions, the class 
is typically defined as 'all present and future applicants'" (p. 
518) . Logic also dictates, she says, that results for a narrowly 
defined group at a particular time have little or no relevance for 
long-term policy decisions. Second, she says that Cohn is concerned 
with passing rates among "qualified" applicants and assumes that 
selection procedures are valid. But "the selection procedures 
themselves are at issue if they have the effect of 
disproportionately excluding a group protected by the Act for 
whatever reason .... Title VII analysis begins with the 
egalitarian assumption that abilities are equally distributed among 
groups in our society" (p. 525) except in certain special skill 
areas (e.g. doctors, lawyers, teachers). Last, she argues that 
Cohn's use of correlation analysis ignores the Supreme Court 
formulation of Bonafide Occupational Qualification and the decision 
that correlations are insufficient proof (p. 533); Cohn's partial 
correlation analysis assumes that job qualifications have already 
been established — but they must be litigated. 

Cohn (1980b) says Shoben is wrong in considering "all present 
and future applicants" as a population. "The legally defined group 
simply cannot be described in quantitative terms" (p. 540) . Cohn 
rejoins that the four-fifths rule was the issue in his paper — not 
the replacement of validation studies of selection procedures. His 
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examples were geared to show that the 4/5 rule could be violated 
whether or not there was job relatedness of selection procedures. He 
argued that the rule was poor because there might be several causes 
of its violation. 

Cohn seems to argue that Shoben misunderstood the intent of his 
example. Cohn seems to be saying that the courts' efforts to 
q...ntify have been off -target. He notes that problems of 
operational ization of concepts occur in all fields of social 
inquiry. 
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III Educational Researchers 
^ Gardner, E. (1982). Some aspects of the use and misuse of 

standardized aptitude and achievement tests. In Wldsor and Gardner 
(Eds.), Ability Testing II (315-334). Washington^ National Academy 
Press . 

Gardner cites some principal misuses of aptitude and 
achievement tests (p. 323). Among t:hem are failure to report to 
^ parents in a meaningful way; misinterpreting measures of changes 

over a period of time by comparison of results from tests dissimilar 
in content or norming procedure; use of grade equivalents for 
profiling achievement tests. In essence, he concludes, "the problems 
facing measurement today and throughout this decade are primarily 
political rather than technical in nature" (p. 315) . 

Findley, W.G. (1979). Civil rights and elementary/ secondary school 
measurement. New Directions for Testing and Measurement . 1, 79-85. 
Findley discusses measurement practices used in three types of 

i 

civil rights and equal protection cases. They are cases involving 
(1) student group assignments, (2) evaluation of minority programs 
and (3) the setting of minimum competency standards for graduation. 

i 

With regard to the first two, he discusses problems with measuring 
gains by means of grade equivalents and the percentile rank scores 
^ of nationally normed tests. Findley describes the use of percentile 

equivalents of average raw scores for a group coupled with the 
concept of normal growth as maintaining relative rank over a period 

i 
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of time. An example is described showing shifts In percentile 
equivalents by ability level* 

With regard to competency testing, Flndley notes that the two 
central Issues relevant to civil rights are test bias and unequal 
opportunity to meet standards* 

Gable^ R*K* and Iwanlckl, E.F* (1986)* The longitudinal effects of a 
voluntary school desegregation program on the basic skill progress 
of participants* Metropolitan Education. No* 1, Spring 1986* 

The authors analyze longitudinal data for five years from the . 
implementation of Project Concern* The analyses consisted of t-tests 
between the experimental and control groups each year* They found no 
significant differeiices (with one exception) * They concluded that 
there were no major systematic differences between the Project 
Concern group and the central group over the five years* However, 
they argued, this does not imply ineffectiveness of Project Concern* 
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COMPUTER PROGRAMS FOR ANALYSIS OF STUDENT PROGRESS 



Listing of Mainframe Program in SAS (PROC MATRIX) 
Listing of MS-DOS Microcomputer Program in GAUSS 
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//TP9 JOB GLN$KA 

// EXEC SAS,OPTIONS>*NODATE' 

//IN DD DSN«m.KA.GLW.SFMR62,UNIT>DISK,DISP«SHR, 
// VOL-SER-PUB036 

//OUTl DD DSN«W!rL.XA.GLW.RTSS.INDSUM,UNIT«DISK,DISP«(NEW,CATLG), 
// VOL-SER-PUB036 , DCB« ( RECFM^FB , LRECL'BO , BLKSIZE«6320 ) , 
// SPACE-(TRK,(2,1),RL5E) 
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PROGRAM: TIMEPATH (VERSION 9) 

WRITTEN BY 

JOHN B. NILLETT AND GARY L. WILLIAMSON 

VERSION 9 MODIFIED AND ADAPTED BY GARY L. WILLIAMSON 
FROM EARLIER VERSIONS 



LINEAR TIME-PATHS ARE FIT FOR EACH INDIVIDUAL AND 
SUMMARY STATISTICS Al^ PRINTED FOLLOWED BY A LISTING OF 
THE ESTIMATE OF RATE OF CHANGE AND R-SQUARE FOR EACH 
REGRESSION. THE ANALYSES ARE COMPLETED FOR INDIVIDUALS 
WHO HAVE SCORES AT ALL TIME POINTS. (AT THE 
BEGINNING OF THE PROGRAM, PROC MEANS IS RUN FOR THE ENTIRE 
DATA SET AND FOR THE REDUCED SET OF INDIVIDUALS HAVING 
ALL SCORES, FOR THE SAKE OF COMPARISON.) 



CASES WITH HIGH RESIDUAL SUMS OF SQUARES ARE 'TRIMMED', 
THAT IS DELETED, BEFORE THE CALCULATION OF MAXIMUM 
LIKELIHOOD ESTIMATES. (FOULKES-DAVIS* GAMMA AND ITS 
STANDARD ERROR ARE BASED ON THE FULL DATA.) THE PROGRAM 
IS SET TO AUTOMATICALLY TRIM THE TOP 3% OF CASES BASED 
ON HIGH VALUES OF 'SSRES*. THE DEFAULT MAY BE CHANGED 
BY RESETTING THE VALUE OF THE VARIABLE ' TRIM ' IN 
THE MACRO STATEMENTS AT THE BEGINNING OF'tHE PROGRAM. 

THIS VERSION OF "TIMEPATHS" ALLOWS FOR THE INCLUSION OF 
CORRELATES OF CHANGE. VARIOUS STATISTICS ARE CALCULATED 
PERTAINING TO SYSTEMATIC INDIVIDUAL DIFFERENCES IN GROWTH. 



THE FOLLOWING ANALYSES APPEAR IN ORDER: 



O 
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* PROC MEAMS FOR THE ENTIRE DATA SET ; 

* PROC MEANS FOR THE SET OF ZMDXVXPUALS HAVING SCORES ; 

* AT ALL TIME POINTS ; 

* PROC MATRIX— THE INDIVIDUAL REGRESSIONS ; 

* (STUDENT ID, RATE AND R-SQUARE ARE PRINTED, ; 

* AUGMENTED BY ACTUAL RAW SCORE DATA. IN ADDITION ; 

* DELTA R-SQUARE IS PRINTED WHERE FITTING A ; 

* QUADRATIC SIGNIFICANTLY INCREASES R-SQUARE ; 

* (ALPHA-.IO)). ; 

* PROC UNIVARIATE ON RATE, R-SQUARE AND SS(RES) ; 

* DIAGNOSTICS AND FULL DATA FOR THE 10% OF STUDENTS WITH ; 

* LOWEST R-SQUARE, THE 10% OF STUDENTS WITH HIGHEST ; 

* R-SQUARE AND THE 10% OF STUDENTS WITH HIGHEST RESIDUAL ; 

* SUM OF SQUARES ; 

* CORRELATIONS AMONG THE SCORES AND RATE ; 

* PLOTS OF RATE VS SCORE AT EACH TIME ; 

* PLOTS OF W VS SELECTED VARIABLES (WHEN CORRELATES OF ; 

* CHANGE, W, ARE INCLUDED IN THE ANALYSES) ; 

* OBSERVED CORRELATIONS FOR THE TRIWIED DATA ; 

* LIST OF CASES DELETED DUE TO HIGH RESIDUAL SUMS OF SQUARES ; 

* MAXIMUM LIKELIHOOD ESTIMATES (BLOMQVIST), FOULKES-DAVIS ; 

* ESTIMATES FOR SYSTEMATIC INDIVIDUAL DIFFERENCES IK ; 

* GROWTH ; 

* • 

* • 



* NOTES: 

* . 

* THE FOLLOWING MACRO DEFINITIONS DEFINE VALUES 

* SPECIFIC TO THIS PARTICULAR EXECUTION OF **TIMEPATHS*' . 

* IN ORDER TO ALTER THE PROGRAM TO EXECUTE USING 

* DIFFERENT VARIABLES, TIMES, OR TRIM FRACTION, THE 

* APPROPRIATE CHANGES MUST BE MADE IN THESE MACRO 

* DEFINITIONS. IT MAY ALSO BE NECESSARY TO MODIFY 

* INPUT STATEMENTS. 



%L£T XLABEL «%STR( 'RTSSQ' 'RTSSIO' 'RTSSll* 'RTSSIZ'); 

%LET "TLABEL~-%STR ( ' 9 ' ' 10 ' ' 11 ' ' 12 ' ) ; 

%LET ~WLABElI»%STR( 'Wl ' •W2 ' ) ; 

%LET 3x1_"RTSS9; 

%LET "x2'«RTSS10; 

%LET ~X3~«RTSS11; 
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111. %LET X4 -RTSS12; 

112. %LET "wr-Wl; 

113. %Lrr 3W2>W2; 

114. %LET HMAMES>%STR(W1 W2); 

115. %LET .TIME.-%5TR(9 10 11 12); 

116. %LET "trim"*. 03 J 

117. %LET "l -OT 

118. %LET ^GWWA.-l; 

119. %LET "out 

120. %LET TEST^CTBS; 

121. %LET TZHES*6RADE; 

122. %LET T_I»9; 

123. %LET T"r«12; 

124. %LET PLOTLBL-%STR(CTBS READING TOTAL SCALE SCORE); 

125. %LET GRAPH-0; 
126. 
127. 

128. OPTIONS MISSING-* *; 
129. 
130. 
131. 
132. 
133. 
134. 
135. 
136. 
137. 
138. 
139. 
140. 

141. %MACRO WITH W; 

142. DATA ALL; INFILE IN; INPUT 

143. «1 ID $ 44-51 SX $ 97 «2 fc XI. 67-69 #6 &.X2 67-69 #10 S. X3 67-69 

144. 113 GIFTED $ 53 «14 (. X4, 67-69 #16; 

145. LENGTH t.Xl. & X2 t_X3 "t X4. 3; 

146. IF SX«'m" THEN't.Wl."!;" 

147. IF SX«'F' THEN l."wi"»0; 

148. IF SX HE 'M' ANd"Sx"hE 'F' THEN i.Wl,«=.; 

149. IF GIFTED«'l' THEN fc_W2_«l; ELSE t_W2_«0; 

150. DROP SX GIFTED; 

151. %MEND WITH W; 
152. 

153. 
154. 
155. 

156. IMACRO W^OUT^W; 

157. DATA ALLT INFILE IN; INPUT 

158. «1 ZD $ 44-51 «2 6 XI 67-69 «6 ( X2 67-69 «10 fc X3 67-69 ' 

159. #14 fc X4 67-69 #16; " 

160. LENGTH fc"xi fc X2 fc X3 fc X4 3; 











MACRO PROCEDURES 
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lei. %MEND W^OUT_.W; 
165. 

lee. 

167. 

lee. 

169. %MACRO READDATA; 

170. %1F «, GE 1 %THEN %M1TH W; 

171. %ELSe""%W_OUT_W; 

172. %MEHD READDATA; 
173. 

174. 
175. 
176. 

177. %MACRO ELIM^W; 

178. %IF fc.L_«0 %THEN %D0; 

179. " i^Wl ■999; 

180. fc2^~«999; 

181. DR0P~C.V7NAMES; 

182. %EKD; 

183. %MEND ELIM^W; 
184. 

185. 
186. 

187. %mac:ro PRT DLT; 

188. %IF C. trim" N£ 0 %THEN %D0; 

189. PROC SORT DATA'DELET; BY SSRES; 

190. PROC PRINT DATA-DELET; ZD ID; DROP IKT; 

191. FORMAT RATb RSQ SSRES DEL R2 F 5.1; 

192. TITLE4 CASES DELETED DUE TO HIGH RESIDUAL SUMS OF SQUARES; 

193. %EKD; 

194. %MEKD PRT DLT; 
195. 

196. 
197. 
198. 

19^, %MACRO DATAFULL; 

200. %IF & TRIM. HE 0 %THEK %D0; 

201. PROC SORT DATA-RESULTS; BY ZD; 

202. PROC SORT DATA«TRIMIND; BY ID; 

203. DATA FULL; MERGE RESULTS TRIMIND; BY ID; 

204. IF TRIMIHD«0 THEN TRIMD*'*'; IF TRZMIND-l THEN TRIMD«'T' ; 

205. DROP TRIMIND; 

206. %END; 

207. %ELSE %D0; 

208. DATA FULL; SET RESULTS; 

209. %END; 

210. %MEND DATAFULL; 
211. 

212. 
213. 
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%MACRO plotdat; 

%IF Si TRIM NE 0 %THEM %D0; 

PROC SORT DATA«FULL; BY DESCENDING TRXMD; 

PROC PLOT DATA-FULL; PLOT RATE*(fc_Xl. I.„X2. S._X3^ «,,X4„)«TRIMD 
TXTLE4 ; 

%END; 

%ELSE %D0; 

PROC PLOT DATA«rULL; PLOT RATE*(fc_Xl. fc.X2. fc_X3_ fc.X4_); 
TITLE4 ; 

%END; 
%M£KD PLOTDAT; 



%MACRO PLOTW; 

%IF fc_TRIM_ NE 0 %THEN %D0; 
PROC PLOT DATA'FULL; 

PLOT (RATE R5Q S> XI t_X2_ fc_X3, fc_X4_)*(fcWNAMES)«'TRIMD; 
UND; - - - - - 

%ELSE %D0; 

PROC PLOT DATA-FULL; 

PLOT (RATE RSQ fc.Xl^ i,X2, fc_X3_ S._X4_)*(fcWNAMES) ; 
%END; - - - - - 

%M£ND PLOTW; 



%MACRO trxmcorr; 

%IF S. TRIM NE 0 %THEN %D0; 

PROC CORR MOPROB NOSXMPLE DATA*T_DATA; 

VAR INT RATE fc_Xl_ fc_X2_ t_X3. t.X*-? 
T2TLE4 OBSERVED CORRELATIONS ON TRIMMED DATA; 

%EKD; 

%MEND TRIMCORR; 



%MACRO TRMCORW; 

%IF S. TRIM NE 0 %THEN %D0; 

PROC CORR NOPROB NOSIMPLE DATA«T_DATA; 

VAR fcWNAMES INT RATE Xl_ fc_X2_ fc_X3_ «'«X4_; 
TITLE4 OBSERVED CORRELATIONS ON TRIMMED DATA; 

%END; 

%MEND TRMCORW; 



%MACRO OUTPUT; 
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270. 

271. 

272. 

273. 

274. 

275. 

276. 

277. 

278. 

279. 

280. 

281. 

282. 

283. 

284. 

285. 

286. 

287. 

288. 

289. 

290. 

291. 

292. 

293. 

294. 

295. 

296. 

297. 

298. 

299. 

300. 

301. 

302. 

303. 

304. 

305. 

306. 

307. 

308. 

309. 

310. 

311. 

312. 

313. 

314. 

315. 

316. 

317. 

318. 

319. 



ZND SUM-BETA | SSREG | | SSRES | | SSTOT | | RSQ 
Bmr |5SREG2||SSRES2||SSTOT2| 
DEL R2||F||SXGHIF||6AMHA l||TR 



R5Q2| I 
MIND; 



CC«'1HT1' 'RATE' 'SSREG' 'SSRES' 'SSTOT' 'RSQ' 

•INT2' 'Bl HAT' 'B2 HAT' 'SSREG2' 'SSRES2' 'SST0T2' 'RSQ2' 
•DEL_R2' 'SK-MIF' 'GAMMA^I' 'TRIMIKD'; 

OUTPUT IND SUM ROWNAME'ID COLNAME*CC 
OUT-IND_SUM(RENAME*(ROW*ZD) ) ; 
%MEND OUTPUT; " 



** 

** END MACRO PROCEDURES ** 
** *• 

9 

i 

* BEGIN PROGRAM EXECUTION ; 

* i 

I 



%READDATA 



TITLEl STUDY OF STANFORD AND THE SCHOOLS; 
TITLE2 WASHINGTON HIGH DATA, CTBS 1980-1983; 



PROC MEANS; 

TITLE4 DESCRIPTIVE STATISTICS USING ALL CASES; 
DATA SCORES; SET ALL; 
%ELIM W 



IF L «0 THEN DO; 

IF (I XI NE . & S>.X2 NE . S. X3 NE . C 
IF (OC HE 0 t HE 0 fc S.~X33 NE 0 fc 

END;" 

IF S> L GE 1 THEN DO; 



X4 
X4" 



NE . ); 

HE 0); 
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323. 

324. 

325. 

326. 

327. 

328. 

329. 

330. 

331. 

332 • 

?33. 

334. 

335. 

336. 

337. 

338. 

339. 

340. 

341. 

342. 

343. 

344. 

345. 

346. 

347, 

346. 

349. 

350. 

351. 

352. 

353. 

354. 

355. 

356. 

357. 

358. 

359. 

360. 

361. 

362. 

363. 

364. 

365. 

366. 

367. 

368. 

369. 

370. 

371. 

372. 



IF (C. XI NE . S. & X2 ME . & & X3 NE . & lk.X4. HE . S. 

S. Wl ~NE . fi. C. W2 'N£ 
IF (i Xl NE 0 S. i X2 NE 0 t. fi. X3 NE 0 C. tt.X4. NE 0); 
END;" 

PROC MEANS; 

TITLE4 DESCRIPTIVE STATISTICS USING NON-MISSING, N0N-2ER0 CASES; 



* 
* 

* 
* 
* 



THIS FIRST SECTIOK CALCULATES QUANTITIES 
THAT ARE NECESSARY FOR THE LISTINGS AND 
DESCRIPTIVE STATISTICS PERTAINING TO THE 
UNTRIMMED DATA SET. 



PROC MATRIX FUZZ; 
FETCH TEMP ROWNAME 
TIME « t_TIME_; 



NC«NCOL(TEMP) ; 



ID; *R£AD DATA INTO MATRIX: NP X (VT+t I ); 
•THE USER MUST GIVE VALUES; 
*0F TIMEPOINTS HERE ; 

•NUMBER OF COLUMNS— WAVES ♦ OMEGAS; 



IF & L GE 1 THEN DO; 

OM£GA«TEMP(,(NC-fc_L_*l):NC); •PEELS OFF THE OMEGA'S— NP X S._L_; 



TEMP«TEMP ( , 1 : (NC-«._L_) ) ; 

END; 

NT«NC-6_L_; 
NP«NROW(TEMP) ; 
IN_TIME«T1ME(1,1); 
TIME«TIME- J (1 , NT , IN_TIME ) ; 



•TEMP NOW CONTAINS ONLY THE X'S ; 
* NP X NT; 

•NUMBER OF TIMEPOINTS; 
•NUMBER OF PEOPLE; 
•INITIAL TIME ; 

•ADJUST TIMES TO START AT ZERO; 



T«J(NT,l,l)i 
T2=J(NT,1,1) 

X«TEMP'; 



TIME'; •THE TIME MATRIX; 

ItIME' I |(TIME'#«2); 



•ORIENTS THE DATA MATRIX (NT X NP); 



• 



376. 
377. 
378. 
379. 

3e0. • CALCULATE REGRESSION ESTIMATES AND R-SQUARE VALUES 
381. 

382. * * 

383. 

384. BETA«INV(T'«T)«(T'*X); 

385. BETA2«INV(T2'*T2)*(T2'*X); 
386. 

387 , SSREG«VECDIAG(BETA ' *T ' *X ) - ( VECD1AG(X ' ( , ♦ ) •X( ♦ , ) ) #/NT ) ; 

388 . SSREG2«VECDIAG(BETA2 ' *T2 ' *X) - ( VECDIAG(X ' ( , ♦ ) *X( ♦ , ) ) #/NT) 
389. 

390 ! SSTOT«VECDIAG(X'*X)-(VECDIAG(X' ( ,4)*X(*, ))#/NT) ; 
391. SST0T2«VECDIAG(X'*X)-(VECDIAG(X'(,*)*X(+,))#/HT); 
392. 

393. RSQ«((SSREG#/SSTOT)#100); -COL VEC. OF R-SQUARE; 

394. RSQ2»((S5REG2«/SSTOT2)«100); 
395. 

396. BETA«BETA'; "COL. VEC. OF ESTIMATES (NP X 2); 

397. BETA2*BETA2 ' ; 
398. 

399. DEL R2«RSQ2-RSQ; 
400. 

401. F«ABS(DEL.R2«/((J(NP,l,100)-RSQ2)«/(NT-3))); 
402. 

403. SIGNIF«J(NP,l,l)-PR0BF(F,l,NT-3); 
404. 

405. SSRE5«SST0T-SSREG; *SSRES IS A VECTOR OF RESIDUAL SUMS OF ; 

406. *SQUARES (NP X 1) ,* 
407. 

408 . SSR£52«SSTOT2>SSREG2 ; 
409. 

410. * * * •** 

411 . * 

412*. * CALCULATE FOULKES-DAVIS GAMMA AND SE(GAMMA) 

413. * 

414. *** * ' 

415. 

416. IF t_GAMMA_«l THEN DO; 
417. 

418. Tl-MIN(TIME); 
419. 

420. T2«MAX(TIME); 
421 . 

422! PHI«(I(NP)'^(J(NP,l,l)*BETA(l:NP,l)')-(BETA(l:NP,l)*J(l,NP,l)))*/ 
423. 

424. (l(NP)4(BETA(l:NP,2)*J(l,NP,l))-(J(NP,l,l)*BETA(l:NP,2)')); 
425. 



^ O U 



429 . PH1«-^ ( ( PHI>«T1 ) & ( PHI<«T2 ) ) ; 
430. 

431, PHI«PHI-DIAG(PH1); 
432. 

433. GAMMA_I»PHI(1:MP,*)#/(NP-1)J •MATRIX OF GAMMA^I? 

434. 

435. 

436. GWWA«SUM(GAMMA_1)#/MPJ •POULKES-DAVIS GAMMA? 

437. 

436. V_GAMMA-SSQ(GAMMA_I-J(NP, 1, GAMMA) )#/(NP-l); 
439. 

440. STER_GAM«SQRT(V_GAMMA#/HP); 
441. 

442. FREE PHI; 

443. 

444. 

445. END; 

446. ELSE DO; 

447. GAMMA I'J(NP,1,999) i 

448. GAMMA«999; 

449. STER_GAM*999; 

450. end;"" 

451. 

452. ^ ^^^^^ 

453. ; 

454. * THE TIMEPATH ESTIMATES AND R-SQUARES ARE STORED IN A J 

455. * MATRIX CALLED 'RESULT' ALONG WITH THE RAW DATA. J 

457. 

458. IF fc L GE 1 THEN 

459. RESULT«0MEGA| |beta(,2) | |rsq| |del.r2|1f| |signif| |temp; 

460. ELSE .... 

461. result«beta(,2) | |rsq| |DEL_R2| |f| |SIGNIF| |TEMP; 

462. ^ 

464. * CREATE MATRICES CONTAINING TOP AND BOTTOM TEN PERCENT i 

465. * OF CASES BASED ON RSQ 

467. 

468. R-RANK (RESULT (,t L 42)); -ASSOCIATE RANKS WITH RSQ VALUES? 

469. T0PRANK«FL00R(.9iNP)? "CALCULATE NUMBER OF CASES IN TOP 10% ? 

470. B0TRANK«CEIL(.1#NP)? •CALCULATE NUMBER OF CASES IN BOTTOM 10% ? 
471. 

472. TR«L0C(R>«J(NP,1,T0PRANK))? * INDICATOR MATRIX FOR ROWS OF ? 

473. • 'RESULT' THAT HAVE R-SQUARES ? 

474. * IN THE TOP 10% J 
475. 

476. BR=L0C(R<«J(NP,1,B0TRANK))? * INDICATES ROWS WITH R-SQUARES ? 
177. • IN BOTTOM 10% ? 



mi i • 

478. 




482. * * ; 

463. * SELECT OUT APPROPRIATE ROWS OF 'RESULT' AND THE ; 

484. * CORRESPONDING MATRIX OF ID NUMBERS } 

485. * • « 

486. 

487 . TOPTEN«RESULT(TR, ) J TOPSTU«ID(TR, ) J 

488. BOTTEN-RESULT(BR, ) ; BOTSTU-XD(BR, ) ; 
489. 

490. IF fc^L, GE 1 THEN 

491. CC-ft^WLABEL 'RATE' 'RSQ' 'DEL R2' 'P' 'SIGNIF' 

492. fc"XLABEL"j 

493. else" 

494. CC-'RATE' 'RSQ' 'DEL R2' 'F' 'SIGNIF' 

495. fc.XLABEL.; 

496. OUTPUT TOPTEN COLNAME*CC ROWNAME-TOPSTU OUT*TOP(RENAME«(ROW-ID)); 

497. OUTPUT BOTTEN COLNAME-CC ROWNAME-BOTSTU OUT-BOT(RENAM£-(ROW«ID) ) ; 
498. 

499. 

500. 

501. • J 

502. * TOP TEN PERCENT OF RESIDUAL SUMS OF SQUARES ; 

503. * I 

504. * 

505. 

506. IF & L GE 1 THEN 

507 . RESULT-OMEGA | | BETA( , 2 ) | | RSQ | | SSRES | | DEL R2 | | F | | SIGNIF | | TEMP; 

508. ELSE 

509. RESULT«BETA(,2)| |RSQ| |SSRES| |DEL R2| |f| |SIGNIF| |TEMP; 
510. 

511. R2«RANK( RESULT (,S. L ♦3))j •ASSOCIATE RANKS WITH SSRES VALUES; 
512. 

513. TRV«L0C(R2>«J(NP,1,T0PRANK)); 'INDICATOR MATRIX FOR ROWS ; 

514. *0F 'RESULT' WITH TOP 10% ; 

515. *0F RESIDUAL SS ; 
516. 

517. T0PTRV«RESULT(TRV,)J *SELECT APPROPRIATE ROWS OF 'RESULT'; 
5X8. 

519. TRVSTU«ID(TRV, ) ; *SELECT CORRESPONDING STUDENT ID'S ; 

520. 

521. IF fc_L_ GE I THEN 

522. CCl«i_i5LABEL 'RATE' 'RSQ' 'SSRES' 'DEL R2' 'F' 'SIGNIF' 

523. fc^XLABEL^; 

524. ELSE " 

525. CC1« 'RATE' 'RSQ' 'SSRES' 'DEL R2' 'F' 'SIGNIF' 

526. S. XLABEL ; 
527. 

528. OUTPUT TOPTRV C0LNAME«CC1 ROWNAME«TRVSTU 0UT«RESVAR(RENAME«(R0W«1D) ) ; 

529. 

530. 

531. *•*•* * * • 



ERIC 
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535. 
536. 
537. 
53B. 
539. 
540. 
541. 
542. 
543. 
544. 
545. 
546. 
547. 
548. 
549. 
550. 
551. 
552. 
553. 
554. 
555. 
556. 
557. 
556. 
559. 
560. 
561. 
562. 
563. 
564. 
565. 
566. 
567. 
568. 
569. 
5'0. 
E/1. 
572. 
573. 
574. 
575. 
576. 
577. 
578. 
579. 
580. 
581. 
582. 
583. 
584. 



* MODXPy R-SQUAK£ VSCTOR TO PRINT ONLY SiGNIFICAKT RSQ 

* AND OUTPUT TO 'RESULTS* 



NDEL.R2>DEL.R2#(SIGHIF<>J(NP,1, .10) ) } 
IP & I Gl i THEN DO; 

RESULT2-0MEGA| |bETA(,2) I |R5Q| |ssres| |ndel R2| |temp| Iganma I; 
C >& WLABEL 'MITE' 'RSQ' 'SSRES' 'DEL R2'^ 

CxLABElI 'GAMMA I' J 
END;" 

IP & L >0 THEN DO; 

RESULT2«BETA(,2) | |RSQ| |5SR£S| |NDEL R2 | |TEMP| |GAMMA I; 
C ■ 'RATE' 'RSQ' 'SSRES' 'DEL R2' " 

li.Xl«lBEL. 'GAMMA_I'; 
END; - - - 

OUTFirr RESULT2 COLNAME-C ROWNAM£»ID 0UT«R£SULT5 (RENAME- (ROW- ID) ) ; 



** 
** 

** 
** 
** 

«* 
** 



* 
* 
* 
* 

* 
* 
* 



THIS PART OP THE PROGRAM CALCULATES QUANTITIES 
THAT APPEAR ON THE MAXIMUM LIKELIHOOD ESTIMATES 
SUMMARY PAGE AT THE END OF THE OUTPUT. 

THESE QUANTITIES ARE BASED ON THE TRIMMED 
DATA. 



* 
* 

•I 
* 



FIRST: TRIM OUT CASES WITH HIGHEST RESIDUAL 

SUMS OF SQUARES (THE FRACTION IS GIVEN BY THE 
VARIABLE 'TRIM') AND RESET THE BASIC VARIABLE 
VALUES HFEDED FOR THE ANALYSES ON THE REMAINING 
DATA. THE DELETED DATA IS SAVED SO THAT A REFERENCE 
LIST MAY BE PRINTED. 
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589. 

590. IF t._TRIM "0 THEN DO} 

591. TrImIHD«J(NP,1,0)| 

592. %OUTPUT 

593. IF li^L_ CE 1 THEN 

594. RETAIN>OMEGA| |BETA| |RSQ| |S5RE5| |DEL R2| |F| |SIGHIF| |TEMP; 

595. ELSE 

596. RETAIN-BETA| |RSQ| |SSRES| |DEL R2| |F| |SIGNIF| |TEMP; 

597. GOTO MLE; 

598. END; 
599. 

600. TRIM'S. TRIM i *A CONSTANT INDICATING THE PERCENT OF CASES } 
501. " *T0 BE TRIMMED BECAUSE OF HIGH SSRES— STATE } 

602. *IN MACRO AT BEGINNING OF PROGRAM, USING } 

603. ^DECIMAL FORM. (I.E., .03 MEANS 3 PERCENT) } 
604. 

605. TRIMRANK-NP-ROUND(TRIM«NP)-t>l; "THE CUT NUMBER— CASES WITH } 

606. "RANK » TRIMRANK WILL BE ; 

607 . *DELETED | 
608. 

609. IF fc L GE 1 THEN 

610. DATA«OMEGA| |BETA| |RSQ| |SSRES| |DEL R2| |F| |SIGNIF| |TEMP; 

611. ELSE 

612. data«beta| |rsq| |SSRES| |DEL.R2| |f| |signif| Itemp; 
613. 

614. * 'DATA' CONTAINS THE BASIC DATA TO BE THINNED ; 

615. 

616. RDATA«RANK(DATA(,S. L •^4)); *AS50CIATE RANKS WITH THE SSRES VARIABLE; 
617. 

618. IND1«L0C(RDATA >« J (NP,1, TRIMRANK) ) ; *INDICATOR MATRIX FOR ROWS OF ; 

619. *DATA WITH TRIM FRACTION OF ; 

620. *HIGHEST SSRES ; 
621. 

622. IND2«L0C(RDATA < J(NP,1, TRIMRANK)); * INDICATOR FOR ROWS TO BE ; 

623 . *RETAINED ; 
624. 

625. DELETED«DATA(IND1,); *SELECT HIGHEST SSRES CASES ; 

626. THINSTU'IDdNDl,); *SELECT CORRESPONDING STUDENT ID'S ; 

627. STAySTU-ID(IND2,); 
628. 

629. IF fc.L GE 1 ?HEN 

630. CLl«t_i5LABEL_ 'INT' 'RATE' 'RSQ' 'SSRES' 'DEL R2' 'F' 'SIGNIF' 

631. i^XLABEL.; 

632. ELSE 

633. CL1« 'INT' 'RATE' 'RSQ' 'SSRES' 'DEL.R2' 'F' 'SIGNIF' 

634. I XLABEL ; 
635. 

636. OUTPUT DELETED COLNAME-CLl ROWNAME-THINSTU 

637. OUT-DELET(RENAME«(ROW*ID)); 



0 
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641. 

642. RETAIH-rm(lHD2,); •SELECT CASES FOR 'TRIMMED' ANALYSIS; 
6«3. 

644. OUTPUT RETAIN COLNAME'CLl ROWNAME-STAYSTU OUT-T.DATA; 
645. 

646. * * ^ 

647 . * 

648! * CREATE A PLOTTING VARIABLE TO IDENTIFY 

649. * TRIMMED CASES IN PLOTS 

650. * 

651. • * 

652. 
653. 

654. TRIMIND-J(NP,1,0); 
655. 

656. TRINXND1-J(NP,1,1); 
657. 

658. TRIMIND(IKD1,)«TRIMIND1(IND1,); 
659. 

660 . CNAME" ' TRIMIND ' ; 
661. 

662. OUTPUT TRIMIND ROWNAME-ID COLNAME-CNAKE 

663 . OUT«TRIMIND(R£NAME"(ROW«ID) ) f 
664. 
665. 

666. * * 

667. * 

668. * OUTPUT INDIVIDUAL SUMMARY FILE 

669. * 

670. * •*** 

671. 

672. %OUTPUT 

673. 

674. 

675. 

676. *. 

677. * i 

678. * RESET KEY DATA VARIABLES TO VALUES TO BE ; 

679. * USED FOR THE TRIMMED ANALYSIS ; 

680. * i 

681. * * 

682. 

683 . BETA-RETAIN ( , t.L^^l l t.L_-^2 ) ; 

684. RSQ-RETAIN(,t L_*3)J * 

685 . SSRES«RETAIN (7fc.l.*4 ) ? 

686. TEMP«RETAIN(,fc L 4B:6.L^*7*NT) ; 

687. IF Ji L GE 1 THEN OMEGA«RETAIN( , 1 1 6_L_) J 

688. NP«Ni0W(TEMP); "NUMBER OF CASES LEFT AFTER TRIMMING; 
689. 

690. 
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694. 

695. FREE DATA RDATA ZKDl ZHD2 DELETED THZNSTU } 

696. 

697. 

698. 

699. 

700. MLE: * • * 

701. • I 

702. * ZNTERMEDZATE STATISTICAL QUANTITIES NECESSARY FOR «* 

703. * THE CALCULATION OF BLOMQVIST ESTIMATES } 

704. * t 

705. * NOTE THAT THE TIME VECTOR IS RECENTERED SO THAT IT STARTS ; 

706. * AT TIME ZERO. THIS IS DONE TO FACILITATE THE APPLICATION ; 

707. * OF BLOMQVIST 'S FORMULAE. SUBSEQUENT ESTIMATES FOR THE ; 
706. * COVARIANCE MATRIX FOR ESTIMATED INITIAL STATUS AND RATE ; 

709. * OF CHANGE INCLUDE ADJUSTMENT BACK TO THE ORIGINAL TIME ; 

710. * SCALE (WHERE GRADE NINE IS DEFINED AS INITIAL STATUS). ; 

711. * T ZERO IS SIMILARLY ADJUSTED LATER. ; 

712. • ; 

713. 
714. 

715. TIMEPTS«T(,2); •ADJUST TIMES TO START AT 0; 
716. 

717. TBAR»TIMEPTS(*,l)t/HT; *MEAN TIME; 

718. 

719. TSQ«TIMEPTS#«2J 

720. TSQ*TSQ (♦,!), • 

721. SST«TSQ-(NT<fTBAR#TBAR); •SUM OF SQUARES FOR TIME; 
722. 

723. 
724. 

725. BMEAN-BETA(f,)«/NP; •ROW VECTOR OF MEAN EST'D. IN. ST. t. RATE; 

726. 

727. BETA-BETA- J (NP, 1,1 )*BM£AN; *DEVIATION MATRIX FOR PARAMETER ESTS. 

728 . SSBETA-BETA ' •BETA ; 

729. COVBETA-SSBETA«/NP; •MLE OF THE COVARIANCE MATRIX OF 

730. •ESTIMATED INITIAL STATUS (GRADE NINE) 

731. •AND RATE OF CHANGE 
732. 
733. 
734. 
735. 

736. * 

737. • 

738. • BLOMQVIST ESTIMATES OF COVARIANCE MATIX OF INITIAL 

739. • STATUS AND RATE OF CHANGE 

740. • 

741. • • 

742. 

743. ERRVAR«SSRE5( + ,)<l/(NP«(NT-2)); •POOLED ERROR VARIANCE; 
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747. 

748. VAR1HST«C0VBETA(1, l)-( (ERRVAR#TSQ)#/(SST#KT) ) } 
719. 

750. V TRATE»COVBETA(2,2)-(ERRVAR#/SST); 
751. 

752. COVISTR«C0VBm(2, 1)*(ERRVAR#TBAR#/SST) ; 

753. 
754. 
755. 

756. * * * * ** 

757 • * 

753! • ESTIMATION OF T-2ER0, VARIANCE OF KSI(TO), AND THE 

759. * SCALING FACTOR. 

760. • 

761. • •* " ••••••••••••••• 

762. 

763 . T_ZEROc- ( COVISTR#/V^TRATE ) ; 
764. 

765. V KSI,T0«((VARINST#V_TRATE)-(COVISTR##2))#/V_TRATE; 
766. 

767 . SCAL_FAC«SQRT( V_KSI_TO#/V_TRATE ) ; 
768 . 

759! SEM_RATE«SQRT ( COVBETA (2,2) -V_TRATE ) ; 
770. 
771. 

772. • * ***"*' 

773. • 

774. • ESTIMATION OF THE VARIANCE OF KSI(T) AND THE 

775. • COVARIANCE OF KSI(T) AND T_RATE. 

776. • 

777. * ** ' 

778. 

779. T T2ER0«TIME'-J(NT,1,T_2ER0); 
780. 

781, C KSITR «J(NT,1,V_TRATE)#T^T2ER0; 
782. 

783. V KSI_T«J(NT,1,V.KSI.TO)4(J(NT,1,V_TRATE)#(T.TZERO##2)); "NT X 1} 
784. 

785. * * * •••••• 

786 • * 

78?! • ESTIMATION OF THE REGRESSION COEFF AND CORRELATION OF 

788. • T_RATE ON KSI(T). 

789. • 

790. 

791. 

792. B_TRKSI«C_KSITR4»/V_KSI_T; 

793. ~ ~ 

794! R TRKSI«C.KSITR<I/SQRT(J(NT,1,V_TRATE)#V_KSI_T); 

795. 
796. 
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BOO. 

801. 

802. 

803. 

804. 

805. 

806. 

807. 

808. 

809. 

810. 

811. 

812. 

813. 

814. 

815. 

816. 

817. 

81B. 

819. 

820. 

821. 

822. 

823 < 

824. 

825. 

826. 

827. 

828. 

829. 

830. 

831. 

832. 

833. 

834. 

835. 

836. 

837. 

836. 

839. 

840. 

841. 

842. 

843. 

844. 

845. 

846. 

847. 

846. 

849. 



« 
* 
* 



t 

FZHDING OBSERVED VARIANCE AND BETWEEN-WAVE CORRELATION ; 
MATRIX. ; 



X-TEMP; 

XBAR>X(4,)«/NP; 
XDEV-X- J ( NP , 1 , 1 ) *XBAR ; 
SSX-XDEV'*XDEV} 
COV X-SSX«/(NP)} 
VAR''x-VECDIAG(COV X); 

std"x«sqrt(var X)T 

CORR x«cov x«/Tstd X*STD X')J 



* DIMENSIONS (NT X NT); 
•NT X Ij 



ESTIMATION OF RELIABIim OF TRATE-HAT AND OF X(I). 



REL_TR«V_TiJ^TE# /COVBETA (2,2); 

REL_XT«V,KSI_T#/VAR_XJ *BASED ON THE SAMPLE—NT X 1 i 
R„KSI«CORR_X#/SQRT(REL_XT*REL,XT'); *TRUE SCORE CORR. MATRIX? 
R_Kr- ^_KSI-DIAG(R^KSI)*I(NT); *INSERT I'S ON THE DIAGONAL? 



* STORING THE BLOMQVIST ESTIMATES 



T,ZER0,S«T_2ER0 * IN^TIME; 

R1«T_2ER0,S//SCAL^FAC// V_TRATE// V_KS I_TO//REL_TR//SEM,RATE j 

R1A«GAMMA//STER_GAM ; 

R2-R_KSI 

R3«V^K5I^T • //VAR^X ' //REL^XT ' //B.TRKSI ' //R^TRKSI ' t 



2<3 ^ 



OUTPUT Rl OUT-Rl; 
OUTPUT RIA OUT-RIA; 
OUTPUT R2 0UT«R2; 
OUTPUT R3 0UT>R3; 



• COMPUTE STATISTICS RELATED TO SYSTEMATIC INDIVIDUAL 

• DIFFERENCES IN GROWTH. 



IF l._L_"0 THEN GOTO PRT? 

•••••••••• WWWWWWWWWWWWWWWWWWW ^ 

• ADDITIONAL COVARIANCE ESTIMATES INVOLVING J 

* OMEGA I 



p W«(OMEGA - (J(NP,1)M0MEGA(4,)«/NP))); •OMEGA DEVIATIONS ; 

•NP X fc^L.; 

PV « beta! Id W; *DEVIATI0N MATRIX OF PARAMETER VALUES ,* 

* KSI(9), RATE, OMEGA-NP X (2*fc.L,) J 

C__PV«<PV'«PV)#/NP; "MLE OF COVARIANCE MATRIX OF PV; 

V W«VECDIAG(C PV)(3:(2-H. L ),)? •ESTIMATED VARIANCE OF OMEGA'S 

•i^L. X 1; 

C TH W«C PV(3:(2+J. L ),2)i •ESTIMATED COVARIANCE (RATE, OMEGA) J 
" ~ " " * "fc^L^ X 1 VECTOR; 

C KSO W-C PV(3:(2*t„L^),l)j ^ ^^^^^ * 

" ~ -ESTIMATED COVARIANCE (INITIAL STATUS, OMEGA) ; 

njL^ X 1; 

C_KSTO_W«C_KSO_W - ((C_TH_W#COVISTR)«/V_,TRATE) | 

•ESTIMATED COVARIANCE(KSI_T0, OMEGA) ; 
•5. L X 1; 



906. • . 

907. * COMPUTE REQUIRED CORl^LATIOK AND REGFESSION ; 

908. * ESTIMATES ; 

909. • ; 

910. *•••* • 

911. 

912. "ALL QUANTITIES IN THIS SECTION ARE VECTORS— fc L XI; 

913. ■ 
914. 

915. R.TH_W«C_TH^W|f/SQRT(V TRATE#V W); *C0RRELATI0N(RATE, OMEGA) ; 
916. 

917. B„TH_W»C.TH_W^f/V_^W; •REGRESSION COEFF. (RATE, OMEGA) j 
918. 

919. R„KSTO W«C KSTO W#/SQRT(V KSI TO«V W); 

920. " - - - 

921. *C0RRELATI0N(KSI TO, OMEGA) : 
922. 

923. B_KSTO.W«C_KSTO W#/V W; *REGRESSIOH COEFF.(KSI TO, OMEGA) : 
924. 

925. T.U«J(t_L_,l,T.ZERO) ♦ ((V KSI TO#C TH W)#/(V TRATE#C KSTO W)); 

926. - - - - « 

927. *T-UPPER; 
928. 

929. T_L«J(«,_L_,1,T_2ER0) - (C KSTO W#/C TH W); *T-LOWER: 

930. *" " ~ . 
931. 

932. * • , 

933. • J 

934. • COMPUTE WAVE BY WAVE CORRELATION AND 

935. * REGRESSION ESTIMATES 

936. * 

937. •* 

938. 

939. *ALL QUANTITIES IN THIS SECTION ARE MATRICES— fc L X NT; 

940. *EACH ROW CORRESPONDS TO A DIFFERENT W; 
941. 
942. 

943. C X W«(D W'*XDEV)#/NP; 
944. 

945. R„X.W»C^X_W#/SQRT((J(fc_L,,l)*VAR X')#(V WJ(1,NT))); 

946. "* " 

947. •CORR(X,OMEGA) ; 
948. 

949. B_X.W«C^X_W«/(V_WJ(1,NT) ) ; •REGR.COEFF(X, OMEGA) ; 

950. 

951. B_KSI_.W«(B^KST0^W*J(1,NT))4((B TH W»J(1,NT))# 

952. (J(t"L ,1)*T TZERO')); 
953. 

95*' *REGR.COEFF(KST, OMEGA); 

955. 
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959* 

96o! R KSI W"R X H#/SQRT(J(S. L ,1)*REL XT'); 
961. 

962. *BASED ON RELIABILXTY ESTIMATE ; 

963. 
961. 
965. 

966. * * * **** 

967 • * 

968*. * STORING THE ESTIMATES RELATED TO SYSTEMATIC INDIVIDUAL 

969. * DIFFERENCES IN GROWTH 

970. * 

971. • •••• 

972. 

973 . 

974*. T U S«T U ♦ J(fc.L.,l)#IH.TIME; 
975. 

976. T L S«T L ♦ J(«i.L^,l)*IN.TIME; 

977. 

978. 

97 9 ! OMEGAl-R TH_W ' //B_TH.W ' //R.KSTO^W ' //B^KSTO^W ' //T_U_S ' //T_L_S ' ; 
980. 

981. * 6 X t_L_ ; 

982. 

983 . 0MEGA2«R^X__W//R_KS I_W ; 
984 . 

985*. * (2 X i.J,J X NT ; 

986. 

987. 0MEGA3-B X.W//B_KS1_W; * (2 X X NT ; 

988. 

989. OUTPUT OMEGAl OUT-OMEGAl; 

990. OUTPUT 0MEGA2 0UT-0MEGA2; 

991. OUTPUT 0MEGA3 0UT"0MEGA3; 
992. 

993. 

994 . PRT: ; 

995. * i 

996. * NOW THE PRINTING BEGINS } 

997. * J 

998. ••••••••••• 

999. 
1000. 

1001. %DATAFULL 

1002. 

1003. 

1004. SCORES-' '; 

1005. IF DEL R2«0 THEN DEL_R2-.; 

1006. LABEL SCORES-* ID«ID; 
1007. 

1008. 



1012. (MACRO PRT WW; 
1013. 

1014. PROC PRXKT LABEL SPLIT-*} ID ID; 

1015. VAR fcWNAMES RATE RSQ SCORES DEL R2 SCORES 

1016. fc.xl^ fc.X2. fc,X3^ t.X4. 

1017. FORMAT RATE RSQ DEL R2 5.1; -------- 

lOlB. TITLE4 FITS TO IMDIVIDUAL T?MEPATHS; 

1019. PROC UNIVARIATE DATA-RESULTS PLOT NORMAL; ID ID ; 

1020. VAR RATE RSQ SSRES GAMHA^IJ 

1021. TITLE4 ; 
1022. 

1023. * 

1024. * SORT BY RSQ AND PRINT OUT TOP AND BOTTOM 10% 

1025. •••• 

1026. 

1027. PROC SORT 9ATA*B0T; BY RSQ; 

1028. PROC FP.INT DATA-BOT; ID ID; FORMAT RATE RSQ DEL R2 F 5.1; 

1029. TITLE4 FITS WITH LOWEST R-SQUARE; 

1030. PROC SORT DATA*TOP; BY RSQ; 

1031. PROC PRINT DATA-TOP; ID ID; FORMAT RATE RSQ DEL R2 F 5.1; 

1032. TITLE4 FITS WITH HIGHEST R-SQUARE; 
1033. 

1034. 

1035. *•* * 

1036. • 

1037. '* PRINT TOP 10% SSRES 

1038. • 

1039. •*••• • ••••• 

1040. 

1041. PROC SORT DATA-RESVAR; BY SSRES; 
1042. 

1043. PROC PRINT DATA-RESVAR; ID ID; 

1044. FORMAT RATE RSQ SSRES DEL R2 F 5.1; 
1045. 

1046. TITLE4 FITS WITH HIGHEST RESIDUAL SUMS OF SQUARES; 

1047. 

1048. 

1049. 

1050. • 

1051. * CORRELATIONS OF OBSERVED SCORES AND RATE 

1052. • 

1053. 

1054. 
1055. 

1056. PROC CORR NOPROB NOSIMPLE DATA-RESULTS; 

1057. VAR t_Xl_ t_X2_ li_X3_ t_X4_ fcWNAMES RATE; 

1058. TITLE4 OBSERVED CORRELATIONS; 
1059. 
1060. 

1061. %PLOTDAT 
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1065. 

1066. %PLOTW 
1067. 

1066. ?ROC SORT DATA-FULL; BY ID; 
1069. 

1070. • MUST SORT THE DATA BACK INTO ID ORDER AFTER PLOTS; 

1071. 

1072. %TRMCORW 

1073. 

1074. 

1075. %MEND PRT^WW; 
1076. 

1077. %MACR0 PRT.WOW; 
1078. 

1079. PROC PRINT LABEL SPLIT**; ID ID; * 

1080. VAR RATE RSQ SCORES DEL.R2 SCORES fc.Xl_ fc_X2„ fc.X3_ t.X4_; 

1081. FORMAT RATE RSQ DEL_R2 "S.lj 

1082. TITLE* FITS TO INDIVIDUAL TIMEPATHS; 

1083. PROC UNIVARIATE DATA-RESULTS PLOT NORMAL; ID ID ; 

1084. VAR RATE RSQ SSRES GAMMA^I; 

1085. TITLE4 ; 
1086. 

1087. * 

1088. • SORT BY RSQ AND PRINT OUT TOP AND BOTTOM 10% 

1089. • 
1090. 

1091. PROC SORT DATA-BOT; BY RSQ; 

1092. PROC PRINT DATA-BOT; ID ID; FORMAT RATE RSQ DEL_R2 F 5.1; 

1093. TITLE4 FITS WITH LOWEST R-SQUARE; 

1094. PROC SORT DATA-TOP; BY RSQ; 

1095. PROC PRINT DATA-TOP; ID ID; FORMAT RATE RSQ DEL_R2 F l.l; 

1096. TITLE4 FITS WITH HIGHEST R-SQUARE; 
1097. 
1098. 

1099. •••••••• * • • 

1100. • 

1101. • PRINT TOP 10% SSRES 

1102. • 

1103. * " * ' 
1104. 

1105. PROC SORT DATA-RESVAR; BY SSRES; 
1106. 

1107. PROC PRINT DATA-RESVAR; ID ID; 

1108. FORMAT RATE RSQ SSRES DEL_R2 F 5.1; 
1109. 

1110. TITLE4 FITS WITH HIGHEST RESIDUAL SUMS OF SQUARES; 
1111. 
1112. 

1113. * * • * 

1114. • i 
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1118. * CORRELATIONS OF OBSERVED SCORES AND RATE ; 

1119. • i 

1120. * * ***; 

1121. 
1122. 

1123. PROC CORR NOPROB N05ZMPLE DATA*RE5ULTS ; 

1124. VAR ti XI & X2 S. X3 fc X4 RATE; 

1125. TITLE4 OBSERVED CORI^LATZONS; 
1126. 

1127 . %PLOTDAT 
1128. 

1129. PROC SORT DATAtFULL; BY ID; 
1130. 

1131. •MUST SORT THE DATA BACK INTO ID ORDER AFTER PLOTS? 

1132. 

1133. %TRIMCORR 

1134. 

1135. 

1136. 

1137. *MEND PRT_WOW; 

1138. 

1139. 

1140. %MACRO PRINT; 
1141. 

1142. %IF «._L_ GE 1 ITHEN %PRT.WW; 
1143. 

1144. %ELSE %PRT_WOW; 
1145. 

1146. %MEND print; 

1147. 

1148. 

1149. 

1150. %PRINT 

1151. 

1152. 

1153. %PRT DLT 
1154. 

1155. *** * ***** ***; 

1156. * i 

1157. * PRINT MAXIMUM LIKELIHOOD ESTIMATES ; 

1158. * J 

1159. * (ALL QUANTITIES EXCEPT GAMMA AND STER.GAM ARE BASED ; 

1160. * ON THE TRIMMED DATA) " ; 

1161. * i 

1162. * 

1163. 

1164. PROC MATRIX FUZZ; 

1165. TITLE4 & TRIM OF CASES TRIMMED; 

1166. TITLES MAXIMUM LIKELIHOOD ESTIMATES (BLOMQVIST, 1977); 
1167. 
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1171. FETCH FULLDATA DATA-RIA; Cl«' 'j 

1172. 'GAMMA' 'STERGAM'; 

1173. PRIHT PULLDATA COLHAME-Cl ROWNAME«R PORMAT«8.3; 
1174. 

1175. FETCH TRXMDATA DATA^Rl? 01" ' 'j 

1176. R" 'T ZERO' 'SCAL.FAC' 'V.TRATE' 'V^KSI.TO' 'REL^TR' 'SEM^RATE'j 

1177 . PRINT'tRIMDATA C0LNAME-Ci"R0WNAME«r"P0RMAT« 8,3; 
1178. 

1179. FETCH KSI CORR DATA«R2; C2- fc_TLABEL_J 

1180. PRINT KSX^ICORR C0LHAME«C2 R0WNAME-C2 F0RMAT«B.3; 
1181. 

1182. FETCH DATA-R3; 

1183. R2- 'v'kSI.T' 'V_X_T' 'REL_XT' 'B_TRKSI' 'R_TRKSI' ; 

1184 . PRINT I C0LNAME-C2"R0WNAME«R2 F0RMAT«8 . 3 ; 
1185. 

1186. 
1187. 
1186. 

1189. * I 

1190. * < 

1191. * PRINT ESTIMATES RELATED TO SYSTEMATIC INDIVIDUAL i 

1192. * DIFFERENCES IN GROWTH 

1193. * 

1194. * (ALL QUANTITIES BASED ON TRIMMED DATA) 

1195. * 

1196. . 

1197 . 

1198. %MACRO OMEGA; 
1199. 

1200. %IF (c L GE 1 %THEN %D0; 
1201. 

1202. PROC MATRIX FUZZ; 

1203. TITLE4 6 TRIM OF CASES TRIMMED; 

1204. TITLES SYSTEMATIC INDIVIDUAL DIFFERENCES IN GROWTH; 

1205. FETCH DATA-OMEGAl; Cl«li WLABEL.; 

1206. Rl-'R TH W' 'B TH W' 'R KSTO W' 'B_KST0_W 'T^U' 'T^L'J 

1207 . PRINT\ COLNA»ffi-Cl ROWNAME«rI P0RMAT«8 . 3 ; 
1208. 

1209. FETCH CORREL DATA-0MEGA2J C2« fc^TLABEL,; 

1210. R2«'R X W' 'R KSI^W' ; 

1211. DO l-I TO t iZi 

1212. A»l| |t.L_*l| |(2*fc_L_)+l| |(3n_L_)4l; 

1213 . C0RR«C0RREL (A, ) ; 

123 4. PRINT CORR C0LNAME«C2 R0WNAME«R2 F0RMAT«8.3; 

1715. END; 
1216 . 
121/. 

1218. FETCH REGRESS DATA-0MEGA3; 

1219. R3«'B_X^W 'B_KSI_W'; 

1220. DO 1=1 TO t_L_; 



1224. A»l||fc_L.4l; 
X225 . REGR-RiGRESS ( A, ) ; 

1226. PRINT REGR C0LNAME-C2 R0HNAME-R3 FORMAT'S. 3; 

1227. END; 
1228. 

1229. 

1230. %END; 
1231. 

1232. %MEND OMEGA; 
1233. 

1234 . %Or!EGA 

1235. 

1236. 

1237. 

1238. %MACRO PZLEOUT; 

1239. %XF I. OUT -1 %THEN %D0; 

1240. DATA;~SET'znD SUN; FILE OUTl; 

1241. PUT ID $ 1-9 (INTl— RSQ) (F8.3)/ 910 

1242. (ZNT2— RSQ2) (F8.3)/910 

1243. (DEL R2— TRIMIND) (F8.3}; 

1244. ~ %END; 

1245. %MEND FILEOUT; 

1246. %FZLEOUT 
1247. 

1248. 

1249. %MACRO SASGRAPH; 

1250 V %XF &GRAPH-1 %THEN %D0; 

1251. DATA GRAFDAT; 

1252. SET ZND SUM; KEEP ZD SCORE T; 

1253. GOPTZOiis DEVZCE-VEP12FF ; 

1254. SCORE'ZNTl; T-t.T_Z; OUTPUT; 

1255. SC0RE«1NT1 ♦ (W"F-tT_Z)«RATE; T«S.T_F; OUTPUT; 

1256. LABEL SCORE-fcTEST T^&TZrffiS; 

1257. TZTLE .H«3 .F-TRZPLEX &PLOTLBL; 

1258. PROC SORT DATA-GRAFDAT; BY ZD; 

1259. PROC GPLOT DATA-GRAFDAT; 

1260. PLOT SCORE*T«ZD/CTEXT«BLACK NOLEGEND; 

1261. SYMBOL C-BLACK Z«JOZN Rxl30; 

1262. %END; 

1263. %M£ND SASGRAPH; 
1264. 

1265. 
1266. 

1267 . %SASGRAPH 
1268. 
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S) Time Path Program 

Adapted from TIMEPATH (version B> by J.B. Willett and 
Gary L. Williamson, originally translated into GAUSS 
by G. Ghandour. GAUSS version modified and enhanced by 
Gary L. Williamson, Fall 1986 

new ,25000; 

loadp dstatx medianx selif del if recode savB«tod; 



# include menu; 



5)procedure to calculate quant iles 5) 
proc l=quantile(>; .<q> > 

local k, n, qnt , i, v, j, g, p, sv>: , ind>:, sv; 

k=cols(;;>; n=rDws(>:>; qnt=zeros ( k , 1 ) 5 p=q./100 5 
j^trunc ( ( n+l ) . *p ) ? 
g=(^l+l).*p) - jj 

indx=5eqa < 1 , 1 , n ) j 
i=i; 

do until i > k; 
v=submat < >; ? O , i ) 5 
5 v>!=sortc ( ( v^indx ) ) 1 ) 5 
BV=5ubmat ( sv>; ,0,1) 5 
if ( j+1 <= n) and <j >= 1); 
qntCi,13=((l-g) .*BvC g . *svC j + 1 ■. 1 3 ; 

else if < j == n) ; 
qntC i , 13=svC j ,13; 
elseif ( j ==■• 0) ; 
qntCi,l]-5vC 

end if ; 
i=i+l 5 
endo ; 

retp (qnt ) ; 
endp J 

cDend of procedure 5) 

output . i lB=-'^hardcopy reset; 
nc-nt+nl ; 

if d == i; 

}oad dataCnp,nc + l J = -^inf ile; 
elseif d == S; 
open f l=--inf i le j 
data=readr ( f 1 »rip ) ; 
endif ; 

5) 

a The following section must be manually edited prior to running the •]) 
5) program, m order to accomodate your particular set of data 



^ create labels for printing — these must be edited to accomodate the 



stlbll-" rate rsq d^rsq 

Btlbia«" rate rsq ssres d^rsq 

• >tlbl«^" xl x2 >t3 x^'^J 
>:lbie«" Ml >iB H3 >t^*'J 
wlbl«" w"i 

wibie**' w 
tibi«" 

^ 5) if a GAUSS data set is being created, the vector, varnames, ® 
0 should be edited to contain the proper number of x^'s and w^'s 3 

if OLitf i le 1 ; 

if (nl >« 1 and c_of_c >= 1); 

let varnames^id xl x2 x3 x^ v\» rate rsq ssres gamma^i ; 

• elseif <nl == 0 or c_of_c — 0); 

let varname5=id xl x2 x3 x^ rate rsq ssres gamma^i ; 
end if ; 

end:.f ; 



5>Create print formats — these must be edited whenever the number of 
significant digits in the x's is different from the previous run- 



Edit the first set of masks and fmts if correlates of change are 



included in this run. 
w^s are used. 3 



Edit the second set of masks and fmts if no 



if (nl : 




1 


and c 


_of _c 


let mast 




,93= 1 


1 1 1 


let masJ 


::Sr.l , 10 3 = 


1 1 1 


let fiTiti 


IC9 


> 


31 = 




"*.*] 


If 


II 


7 0 




*] 


If 


II 


1 0 




"*•.*] 


If 


II 


7 3 




*] 


If 


II 


5 1 




*] 


If 


II 


10 1 




"* . *] 


If 


II 


10 S 




"*.*] 


If 


11 


6 2 




*] 


If 


II 


6 E 




"*.*] 


If 


II 


6 2 


• 


let fmtacio,3:= 




"*.*] 


If 


II 


7 0 




"*.*] 


If 


II 


1 0 






If 


II 


7 3 




"*.■»<•] 


If 


II 


5 1 






If 


i> 


7 a 




"*.*] 


If 


II 


10 1 




" *• . * : 


[f 


II 


10 a 




"* . *] 


If 


II 


6 a 




"* . *] 


If 


II 


6 a 






Lf 


II 


6 E 


■ 



end if ; 
if (nl 
let mas 
let mas 
let fmt 
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0); 

I 1 1 1 1 ; 
1 1 1 1 1 1 ; 



= 0 or c 
1 CI ,83 = 
ECl ,93= 

ca,33= 



c«f. 
1 
1 



1 
1 



<= 0 ) J, 
1 1 1. i 
1111 



■I 

1 ; 





*: 


.f 


II 


7 0 




*] 


If 


II 


7 3 




*'. 


If 


11 


5 1 




*; 


if 


II 


10 1 




*] 


If 


II 


10 S 


"*.. 


*] 


Lf 


l» 


6 a 




*] 


If 


II 


6 a 




*] 


Lf 


It 


6 a 



t fmteC9,33= 



"».*lf "73 

"*.»lf " 5 1 
• "*.*lf " 7 S 

"*.*lf " 10 1 

•'*.*lf " 10 S 

"».»lf "68 

"*.«lf "62 

"♦.♦If "62 J 
' endif; 

0^(.^t#^^^^^^^^##^^##*#♦##**♦*♦*********^^*^^♦*^^■«♦*•«♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦S> 

3 end manual changes ^ 
©♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦******************5) 

© peel off observed sccM-es> background variablesj produce descriptive S) 
3 statistics ® 



id=dataC . » 1 H j 
tmp=dataC . » Sinc + 1 D ; 
clear data; 

"Time Path on " namedataj " " , 
dstat>! ( tmp) ; output on; "i.f"5 



11 II m II fl ■ 
9 J 



d Partition data matrix into timepaths and correlates 



5) 



x==tmpC . J 1 :nt3 ; 
if nl l; 

omg=tmpC . ?nt+l s nc 3 > 
endif ; 
clear tmp; 

3 adjust time to ©tart at O 

i_tm— tmC 1 > 1 3 ; 

tm=tm--i tm^ones ( nt , 1 ) ; 



d Create the time design matrices 



t_l=Dnes(nt ? 1 )^tm; 



3 Re-^orient the time path matvix 



5) 



© Calculate regression estimates and r-squares 



3 



bete_l=^solpd(t_l'"'x5t_l^t_l ) ; 
be ta_e=BD Ipd ( t^E ' x , t_E t_2 > ; 



crf=Bumc ( X ) •*sumc ( x ) /nt ; 



• idx=l; 

ssreg_l=2eros ( np f 1 > ; 
ssreg_E-2eros(np ? 1 ) ; 
B5tot=2eros ( np , 1 ) ; 
do while idx <> np ; 
ssreg.l C idx , 1 ]=beta_l C . > idx3 '*t_l "♦xC . , idx3 ; 
I ssreg^SCidx, 13=beta_S[ . ,idx3 '^t^E'^xC . , idx3 ; 

O sstDtC idx , . 3^xi: . , idx3 '*xC • 5 idx3 S 
ERjC idx^idx-H; 2o5 



ssreQ_l*«sreg_l-crf I 

ssreQ_2«ssr eg.e^crf ; 

sstotssstot-^crf ; 

rsq_l»«ssreg_l . /sstot J 
rBq_S«SBr eg_S . /ssto 1 5 

d_rsq«rsq_2-rsq_l 5 

f_v»abs(d_rsq. / (oneB(nps 1 )-rsq_2) )/(nt-3) 5 
f_s»cdff c7f_v>ones< np , 1 ) , (nt-3)*DneB<np $ 1 ) > > 

5Bres_l«5stDt-ssreg_l ; 
S5reB_S«sstot-Bsreg_S; 

clear erf, beta^Ej f.Si f^vs rsq^S, ©sreg^S, ssres 

0Dutput fits to individual regressionsS 

namedata; " ; " 

format /ml /rd 7,2? 

"Fits to individual regresBion^ " ; 



II II • 
9 



if (nl >= 1 and c_of_c >= 1)? 

idlbl$H"Wlbl$+Bt Ibl l$+xlbl ; 
II II J 

resul t~\d'^omg~beta_l nS , . 1 ' ~100 . ♦rsq_i'^100 . *d_rsq'^x 

p=pr intffn<resul t ifnask 1 if mtl ) 5 "\f " 5 

else; 

idlbl*+stlbll$+/.lb]. ; 

II II • 

re5ult"--id'^beta_l [2, . D "^1 00 . -J^-rsq^l '^lOO - *d_r sq'^x " ; 
p=printfm(reEult ^maskl jfmtl ) ; "\f *' ; 
end if ; 

cDselect top and bottom 10% of cases based on rsq 

sortrslt-Bortc ( result >3) ; 
bo tcut-cei 1 ( . 1 . -^ np ) ; 
topcut=f loor < . 9 - ^np ) ; 
bo tt(?n-sortr5l t C 1 • Dotcut ? . ] ; 
topten=sor trsl t C topcut : np j • 3 ; 

sprint top and bottom lOX based on rsq 5) 

. namedata? ; " " ; 

"Fits with lowest R-square" ; " ; 
if (nl >= 1 and c^of^^c >= 1); 
idlbl$+wlbl$+stlbl l$+>:lbl ; 
else ; 

idlbl*+stlbll*+Nlbl ; 
end if ; 

p=printffn(botten,ma5kl ,fmtl ) ; ''\f*'i 
namedata; " "5 " 

"Fits with highest R-square"; " " ; 
if (nl >==1 and c_of_c >= 1); 
idlbl$+wlbl$-Hstlbl l*+xlbl ; 
el se; 

idlbl*+5tlbll$+)Ubl ; 
end if S 

p«printfm( top tens mask 1 ^f mtl ) ; '*\f " > 

o ^ ^ n 

:nipear botten> toptenj a^ou 



S)Find fltB Mlth highest residual sums of squares 3 

^ clear result* sortrslt; 

if (nl >« 1 and c.of^c >«1 ) I 

result«=id'^omg~beta_lCS, • 3 "'^lOO.^rsq^l'^ssres^l'^lOO. ♦d_rBq'^>s " j 
else; 

result^id'^beta.l CSi . 3 ''^100.*rsq_l'^ssres_l^iOO.*d_rsq'^>( ' ; 
end if ; 

^ sortrsl t»sr.irtc (result > ^) 5 

topres^sortrsl t Ctopcut :np ji . 3 ; " " S 

Sprint top 10% of cases with high SSRES ® 

namedata; " 5 " ; 
^ "Fits with highest residual sums of squares"; " 
if (nl >= 1 and c^of^c 1); 
idlbl*+wlb3*+stlbr2$+ulbl ; 
el se; 

idlbl*-4.s+lbl2$+>ilbl; 
end if ; 

0 p~pr intf m( topresjmaskSjfmtS) ; "\f " ; 

clear topres, sortrslt; 

©calculate? observed correlations 5> 

I if nl — 0; 

m=^>('^'^bc?ta^lCE, • 3 5 
elseif nl > 0; 
m=^>i'~omg'^beta_UE, . 3 ' ; 
endif ; 

fn_bar=meanc (m ) ; 
I m_dev=m-one5 ( np y 1 ) ^(u^b ar ^ ; 
cciv_m==( m_dev''*m_dev ) . /np ; 
sd_m— sqrt (diag (cov_m) ) ; 
CDrr_m-cov_m, / (sd_m-«-sd_m ) ; 
if c_of_c <= 0; 

corr_m= < corr^mC 1 :nt 1 1 : nt 3 ~corr_mC 1 :nt jnc + 13 ) 1 
<corr_mCnc + l 9 1 : nt 3'^corr_mCnc + l jinc + 13 ) ; 

endif; 

clear m_bar> m_dev, cov_mj sd_m; 

Sprint correlatioHB between waves j omega and w 3 

namedata; " i "5 
II 11 • 

'*Cbse)-ved corr el at ions > untrimmed data S 
1} It • 

if (nl >- 1 and c_of_c >= 1); 
>;lbia$^wlbl2$+'^ rate"; 
el se ; 

xlblS$+" rate" ; 

endif ; 

format /ml /rd 7,3; 
corr_m; "\f ; 

5) Calculate Foul kes-Davis gamma and S.E. (gamma) -i) 
id>; = l ; 

phi«zero5 (np ? 1 ) ; 
gam_x==2:eros(np 5 1 ) 5 



while idx 0 np; O***^ 
E]^t"^hi«beta_l ca? id>{3^ones(np, 1 )-beta_lC2!t . 3 ' ; ^^0? 



phi»recod«<phi tetv) ; 

phia(beta_lCl » . : '-beta^l C 1 » idx3»ones(np , 1 ) ) . /phi ; 
I phi».not (Tphi .>«tmCl , 13 ) .and (phi . <»tfnCnt » 13 ) ) ; 
v-05 

phi«recode(phi »eiv) j 
gam_iCid>!> 13=sumc<phi >/<np-l ) » 
id««id>j+l; 
endo? 
I clear phi» e; 

gam^f d=meanc (gam_i ) ; 
5e_gams=5tdc ( gam_i > , /sqrt < np ) ; 

beta_l«beta_l ' ; 

^ S)de5cr ip t ive statistics on rate? rsq? ssres and gamma© 

tmp=beta_l C . ,a3~100*rsq_l~ssrBS_l~gam_i S 

tmeavi^meanc ( tnip ) ; 

t5td=Btdc ( tmp ) ; 
I tmin=minc ( tmp ) ; 

tmax=ma>:c ( tmp ) ; 

tmed^med i anx ( tmp ) ; 

tq l=quant i le ( tmp > 25) ; 

tq3=quantile( tmpj75) ; 

tp. 5=quanti le< tmp j5) ; 
^ tp95=quanti le( tmp »95) ; 

summary=tmean'^tstd~tmin'^tp5~tql'^tmed'^tq3'^tp95'^tma>j ; 

summary=summary j 

clear tmp? tmeanj tstdj tmin? tmax> tmedj tqlj tq3j tp5j tp95; 

Sprint summary statistics for ratsu rsq? ssres? gamma 5) 
I namedata; " "; " "; 

"Descriptive statistics and quantiles"; " "; 

let names=mean std mni p5 ql med q3 p95 max; 

Eummary=names~summary ; 

let masi^Sri ,53=-- 0 1111; 

let fmt3C5,3]= 
I "-*.*s " 8 8 

"*-.*lf "83 

"*.*lf "83 

"•^.*lf "83 

"*.*lf " 8 3; 

" " ; " rate? rsq ssres gamma"; 

\ p=pr intfm (summary 5 masks J fmt3) ; " "? 
clear summary; 

a) print highest and lowest rates and rsq's and corresponding id's 5) 

"Cases with extr-^me values of rate and rsq? identified by id"; " "; 
) "Highest"; " 

e>;trate=beta„l C . lED'^id 5 
ex trsq-rsq_l'^id ; 
extrate=rev<sortc (extrate? 1 ) ) ; 
extrsq=rev( sortc (extrsq ? 1 ) ) ; 

I let ma5k^Cl,^]=l 1 1 l; 
let fmt^fC^f ,33 = 
"*.*lf " 8 3 
"♦.*lf " 8 0 
"♦.♦If " 15 3 
"♦.♦If " 8 0 ; 

I 

0 rate id rsq id"; 

ERIC np < 20; "^'-^ 



p»printfm<e>{tremesiffla»k<»ifmt<») J " "I 

else; 

# hiextrmssextrateCl :10, . D'^OKtroqC 1 : lOi .3 J 
loe«trms»eNtrateCnp-10:np» . 3'*'e«tr5qCnp-10:np » . 3 I 
p»pr intfm(hieKtrms»mask^,fmt^> J " "» 
paprintfm( loe>{trms»fnask^ifmt^) J " "; 

endif* ; 

clear extrdte? extrsq? 'lieHtrmsj loextrms; 

# "Lowest"? "\f"J 

©create gauss data file with relevant variables 3 
if nl 0; 

tpmat=id'^>!~beta_l C . , 23'^r5q_l'^ssres_l'^gam_i ; 
el SB if nl > 0; 

# tpmat='id'^x'^omg'^beta_l C . , S3'^rsq_l'^ssre5_l~Qam_i ; 
endif 5 




# i nc 1 ud e t r i mm i ng 

clear result, c^trm, d^rsq, gam_i 5 id^ rsq.lji r_trm? ssreg^ly ssves^l; 



ntrm : 

# ttinclude psyc.prgS 
output off; 

if Dutf i le 1 ; 

savextod; ^invokes interactive program to Sc.<ve output to 

a disk file. The user supplies the name of the output 
fi^e.M the name of the matrix in memory that is to be 

# stored in the output filen and the name of a character 

variableji which names the columns (i.e-» variables) in the 
matrix. The name of the output file can be chosen arbitrarily. 
The? name of the matrix in memory that will be stored is 
"geussdat". ••varnames" is the character va>~iable 

thiit supplies the variable names for the output file 8 

% endif; 
end 5 

stoptp : ; 
end 5 



# 



• 
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© interactive version of timepath S) 

II II ■ II II a II II ■ II II ■ II II ■ II II ■ 

> s > > > ? 

"This is the Timepath program for fitting straight-line growth"! 

"curves to individual longitudinal data sequences. "; 

"This interactive question and answer session allows you to 

"specify the necessary information for running the program"; 
II II ■ 

"First* you must describe the input file: 

" the matrix loaded will be np<# persons) >; "? 

Kid) + nt(# waves) + nl(# correl . ) " i 

" You must specify the values npt nt > nl when prompted below. 

" The input file may be either an ASCII file (if <6^k) or a "; 

" GAUSS data set- In either case, the"; 

" variables in the file are assumed to be in the following order: 

" id xl >;2 . . . >;T wl wE • . . . 

f) II ■ 

9 

II II a 
9 

"If your data are not of this form, you must exit now and pre-process" 5 
"the data before running this version of Timepath-"; 

M II ■ 

9 

"Are you ready to proceed? (y, n)"; 
proceed^cons; " "; 

if (proceed "n" or proceed "no"); 

goto stoptp; 
endif ; 



"Are your data in an ASCII file or a GAUSS data set? (1==ASCII, S=GAUSS) 
d=con(l,l); " "; 

"Please give a name to your data that can be used in page"; 
"headings. Use ^yO characters or less."; " "; 
"Title for data set:";; 
namedata=cors; " "; " "; 

"Enter the number of observations (i.e.? individuals j cases):*'; 
np=con(l,l); " "; " "; 

"Enter the number of waves of data for each individual;"; 
nt=con(l?l); " "; '» "; 

"Enter the time pointsu e.g. 0 1 S 3j at which measurements were taken: 
tm=con(nt9l ) 5 " "; " "; 

"Enter the number of background variables (w's) in the i nput .f i le : " ; 
nl=con(l,l); ; ; 



if nl O; 
c_of_c=0; 
goto skip; 
evidif ; 



"Do you want to use these background variables to compute correlates"; 
"of change in this execution of Timepatt-^? (l^yes^ 0-no)"; 

« It tl . II n m 

9 9 9 



c of c=cc.n( 1,1)= " " ^ " " 



skip: n 



GO 



SEST COPY AVAILABLE 



ERJQ ciu can trim observations from the data automatically based"? 

....... 



''manually by specifying the id numbers of cases to be trimmed.''} 
"The trimmed cases are included in descriptive growth analyses"} 
^ "but are set aside before computing the psychometric quantities."? 
"What type of trimming do you want to do?"! 
" (O»nonej l»automatic % 2=manual ) " ; 
trm=cDn(l>l); " "; 

if trm 1 5 

^ "Enter the percent of cases to be trimmed (a whole number 0):"; 
p_trm«con(l!il)J "S " "; 

elseif trm =« 2i 

^ "How many cases do you want to trim?"; 
n_trm«con(l,l) J " "; 

"Enter the id number of each case that yoL\ want to trim* followed by"; 

"a carriage return."; 

c=l; 

0 do while c <- n_trm; 
tmp id-con < 1 n 1 ) ; 
if c 1 ; sel ids=tmp id ; 
else; 

sel ids-sel ids 1 tmpi d ? 

end if 5 

0 c^c+l; 

endo ; 
II II • 

end if ; 

H * Do you w< ot to create a GAUSS data set containing the scoress"? 
"estimated rates?* and other information? (l-yer.. » 0=nD ) " ; 
outf ile=con( 1 , 1 ) ? " "; " "i 

i-f outf i le 1 ; 

"When the Timepath output is finished ^ you will automat icc^.lly be"; 
^ ''thrown into a menu-driven program that creates the GAUSS data set."; 

"In order to complete the process j you will have to specify three "; 

"things- Firsts you must give a file name (including disk drive"; 

"specification) where the GAUSS diata set will be stored."; 

"The name is arbitrary. Choose one that is meaningful to you."; 

"Secoridj you must specify the GAUSS matrix in memoryj which contains"; 
^ "the scores and statistical quantities that are to be saved in the**; 

"GAUSS data set. Finally^ you must specify a vector that contains"; 

"the names of the variables (or columns) in the data set. The"; 

"GAUSS matrix arid the vector are determined in the Timepath program?"; 

"and are as follows:"; 
11 II . 

^ "The data matrix to be stored is called ^tpmat',*'; 

"The variable (colunm) names are contained in the? vector? varnames ^ . " ; 

II II M 

9 

"You must remember ^tpmat' and ^varnames'n as you will'*! 
"be prompted for them when you create the GAUSS data set**'; 

•II fl w 

end if ; 



"Please give the name of the file that contains the input data.'*; 

"Use the form? d :f i lename.e>it j where 'd" is the appropriate disk drive. 

•II II ■ 

(ji' nput file: " ; ; 
ERjCf ile=cor»s5 " "5 " "; 
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"Give the name of the output file that will contain the hard copy" 
"report produced by the Timepath program. Be sure to give the dri 
"designation and the full file name."; 

M il ■ 

"output file: " ; ; 
hardcopy=consj " "? " "5 



O n o 



if trm === O; p_trm=0; goto ntrm! 

• elseif trm 25 goto st ; 
end if ; 

if p^trm <= 0; 

goto ntrm; 
endif ; 
#^ p_trm«p_trm. /I 00 J 

n_trm«cint<np.*p_trm) ; 
if n_trm 0; 

""^♦^-N-*^ percentage of trimming too smell — no cases trimmed ***** "5 
goto ntrm? 
endif ; 

# np_o-np ; 
np=np-n_trm; 

c_trm=Esres_l'^5eqa( 1 ^ 1 >np_o ) ; 
c_trm=^sor tc ( c_trm j. 1 ) ; 
s_trm=c_trmCnp+l : np_o j 23 ; 
^ r_trm=c_trmC 1 :np jS"J ; 

St: 

if trm " 2; 



n_trm=^row5(Bel ids ) ; 
np_o-np ; 
^ np=- np-n_trm 5 

fullmat^dataC . , 1 ^'^seqaC 1 , 1 ,np_o> ; 
si = i ; 

do while si <= n_trm; 

e=(ful ImatC .,13 .== (sel idsCsi > 1 D .*ones(n^^_u , 1 ) ) ) ; 
if si 1? 

♦ c_trm«sel if(fullmat,e): 
r_trm==del if (ful Imat >e) ; 

eE=^(r_trmt .,13 (selidsCsi + un . *ones (np^o-si , 1 ) ) ) 5 

else; 

c^trm'^c^trm ! sel if (fullmat je) ; 
r_trm=del if (r_trm jbS) ; 

• if si < n_trm; 

ee=-'^(r_trmt .!il3 (selid5Cr^i + l,13 . *Dnes ( np_o-si » 1 ) ) M 

end i f ; 
end i f ? 
s i =s i 1 ; 
endo ; 

# s_trm^c_trmC . jSH; 
^^_*t^*fT^-r_trmr • "iSD ; 

clear fullmat? c trm? e? eS; 



p_trm=^(n_traK ./np_o ) ; 
# endif S 



5)t rimmed cases S> 
3 

t_?j^«^uhmat ( >[ 5S_trmjiO > 5 
t_beta_l-5ubmat (beta_l *. s_trm,0 ^ ; 
# ^r. ^ssres-submat (ssres_l ns^trm^ 0) 5 




ssregsssubmat < ssreg_l ,s_trm>0) ; 
s^tot^s.ibmat (sstot iS^trm^O) ? 



tirsqis-submat (rBq.2»s_trm,0) ; 
tldrsq»submat (d_rBq>s_trm,0) ; 
t~f_v«Bubmat (f_v,B„trm»0) ; 
t~f Is«submat ( f _s » s_trm 1 0 ) ; 
if c_of_c > O; 

t_omg«Bubmat (omg »B_trfn»0) i 

endif J 

©print cases deleted 5> 
namedata; " "5 " "J 

if trm == 1 ; ^ 

"Cases deleted due to high sums of squared residuals ' » 

elseif trm == SJ 

"Cases deleted manually"; " "» 

end if; 

if (nl >= 1 and c..of_c > 0); 
idlbl*+wlbl$4-5tlbie*4->(ibl; " "; 

else ; 

idlbl*+stlbie*+>;lbl ; " "; 
endif; 

p=printfm(submat(result ,s_trm,0) ,maskE,fmtE) ; " "5 "\f"; 

©reset values of original variables so that they contain 
the cases retained for mle analysis 5) 

>t"Submat (>t!>r_trm jO) ; 
beta_l=submat (beta_l ,r_trm5 0) ; 
ssres_l=submat ( ssres_l jr^trmjO) ; 
.9 

S5reg_l=submat ( ssreg_l ? r_trm ? 0 ) j 
5:stot=submat < sstot ? r_trm ? 0 ) ; 
r5q_l=submat (r5q_l jr.trmjO) j 
rsq_E=submat (rsq_e>r_trm!iO) ; 
d_rsq=submat (d_rsq , r_trm!. 0 ) ; 
f __v=submat ( f _v j r_trm s 0 ) ; 
f _s=submat ( f _s > r_trm s 0) ? 
5)" 

if (nl >= 1 and c_of_c > 0); 

omg='iubmat ( omg ? r^trm » 0 ) j 
endif; 



2R4 



FRir 



9 Blomquist estimation ® 

5> sum of squares for time ® 
ss_tm«sumc( tm. ''2>~nt*\ f meanc( tm) ) .^"£ ) ; 

5) parameter covariance matrix ® 

beta_d==beta_l-- ones(np ? 1 )*meanc (beta_l ) ' ; 
cv_beta-(beta_d \beta_d ) • /np ; 

5) blomquist estimates of covariance matri; of initial status and rate 
of change 

err_var=sumc ( ssres.^.l ) . / (np-x-Cnt-^S ) ) ? 

in5_var«cv_bfcitari ^ i D-< ( err^var . ^suff.ci fmn "-^ ) ) • / ( ss_tm - -K-nt ) ) 5 
trt_var=cvIbetai:E>S3*-(err_var . /ss^tm) j 
ist]^cov=cv_betaCS!i 1 3 + (err_var • -x-meanc ( tm) ./ss^tm) ; 

5) estimation of t-2ero!i variance of ksi(t-2ero) and the scaling 

factor ® 



0 



t_2ero=-l ist^cov . /trt_var ) ; 

vlksi_tO= ( ( ins^var . -N-tr t _var ) < i £ t^cov • "'E ) ) . /tr t_var ; 
Bcl^f ct^sqrt ( v_k5i_tO. / trt^var ) ; 
eem_rate=sqr t (cv_betaC£ j£3-trt_var > ; 

5> estimation of var iance( ksi ( t ) ) and the covar iance ( ksi ( t > ? t_ r ate 



t_tzr=tm-oneB ( nt n 1 ) • -^^t^zero !i 

c_ksitr = (oneB(nt , 1 ) .-^^trt^var ) .*< t_tzr ) ; 

vIkBi_t=(ones(nt 1 ) - ^v^ksi^tO^-ConesCnt ? 1 ) . ^trt^var ) .^i t^tzr . "'^2 ) ; 

.j) estimation of regression coefficient and corr of t_rate on ksi'.t) ^i) 

b_tr ksi=c_kBi tr . /v_k^i_t ; 

r_tr k5i=c_ksi tr . /^.nj t (ones(nt ? 1 ) .-J^trt^var .*\r_kBi_t ) ; 

a observed variance .-nd between-waves correlation — trimmed data S) 



x_bar=meanc ( x ) 5 
><_dev=M-ones (np > 1 )*•>:.. bar ' ; 
cov_>j5= ( >{_dev -N-K^dev ) ./np; 
sd_>; = Bqrt (diag (cov_k > ) ; 
corr^xs^CDv^x./Csd^x-^-^d^x" > 5 

Sprint observed correlation matrix — trimmed 3 

^ namedata; " '^5 

*'Between-wave correlation matrix — trimmed data '* ; 

format /ml /rd 8,3; 

tml = tm'+-i_tm^ones ( nt , 1 ) ; 

tmlM 

" tml'^corr x5 '^R^'^ 
ER^Oobserved variance and reliability of fitted values 



f'_^clev»f its-ones <np , 1 >*f. .bar ' J 
CO v_f i t « < f _dev * *f _dev > . /np J 
varif it«diag<cov_f it ) \ 
relief itav_ksi_t. /v^r.f it > 

$ reliability estimation of trate_hat and x(i) 

rel tr«trt_var ./cv_betai:Sj23 S 
rell>tt«v_ksi„t ./(sci_>!^ S) ; 
relIksi=cDrr_x . /sqrt < rel_,xt*rel_>{t *'' ) 5 
rellk5i»diagrv (rel_ksi lOnesCntil ) ) J 
t_sero_s=t_sero+i_tm; 
3 correlates of change statistics 

if (nl 0 or c_of_c 0)! 

goto XXX » 
end i f 5 

S> additional rovariance estimates involving omega 

omg_dev=omg-ones ( np » 1 ) *meanc ( otuq ) ' » 
pv=beta_d'^omg„dev ; 
c_pv=(pv''*pv) . /np; 
v'_omg=''diag(c_pv) ; 
v_omg=v_omgC3>nl+-S} . 1 \ 
r=sfcqa<3, 1 ,nl) ' ; 
c_th_w=submat <c_pv,r jS) ; 
c_ksO_w=submat <c_pv , r » 1 ) J 

c_kstO„w=c_ksO_w-< (c_th_w .*ist_cov ) ./trt_var > \ 

5) cc- .-relations and regression esti'-.istes 

r_th_w=c_th_w . /sqr t ( tr t_var . *-v„Dmg ) ; 
b..th„w=c_th_w . /v_o(ng ; 

r"_kstO_ w=c_k5tO. w . / sqr t ( v_k«= i _tO . *v_omg ) S 
b_kst(.)_w=c kstO w./v.omg; 

t„».'=t_2eroT*one5(nl , 1 ) + ( ( v_k5i_tO . *c_th_w ) . / ( tr t_ver . * c_kEtO_w ) > ; 
t3l=t_2ero . *ones( nl ? 1 ) - (c_kstO_w.. /c_th_w ) ; 
t_u_B=t_u-«-i_tmil 
tllls=tll+i_tm; 

5) compute wave by wave correlations 
c_x w'=<Dmg_dev''*>; dev^./np; 

r_xIiM=c_xlw./sqrt ( (ones(nl , 1 ) * ( sd_x-£ ) ' ) . * ( v_Dmg*ones ( 1 ,nt; ) ) j 
b~x_w-c_>!_w. / ( v._c«mg-«-Dnes( 1 J nt ) ) ; 

b ksi w=(b kstO 'w«ones( i ,v ';)> + ( <b_th_w«ones( 1 ,nt 5 ) 

(ones(nl !. 1 )*t_t2r ) ) ; 
r_ksi_w=r_x_w. /sqr t < ones < nl , 1 ^ *rpl_xt '' ) j 
X x >; : 

a output code follows here 
formc*^, /ml /rd 8,3; 

" Time. Path or " namedata! 

II II • 

FoLilkes-Davis Tracking Index";" "j 
Gamma: " gam_fd; 

Standard Errc-r (Gamma): " se.gam?" ";" '5 
Blomqvist Maximum Likelihcod Estimates";" "» 
if irr. 0; 

" Percent of cases trimmed:" p_trm.*100; " 

elseif trm 1 ; 
O ■percent of cases automatically trimmsds" p_trm.*100; " "5 

ERfclseif trm »= 2 5 OfiH 
Percent of cases manually trimmed:" p_trm.*100; " "; 



3 



•5> 



tl 



Scaling Factor: " scl.fctS 

True Rate Variances " trt^varl 

" Variance KsiCt<rero ) 3: *' v_ksi_t05 

Reliability of Rate: " rel_tr5 

S. E. Measurement of Rate:" sem^rate;" "I 
tm- tm+ i _tm«ones ( nt » 1 ) > 
format /ml /rd; 

" Disattenuated Correlation Matrix"; 

tm'^rel_ksi ; " "J" "» 

" tm--;" "5 

Var(Ksi) " v_ksi_t'5 

Var(X) " diag(cov_>;) " ; 

Var(X_hat) " var^fit'; 

Reliabil ity <X) " rel^xtM 

Reliab(X J.at) " relt_fit''; 

ERta(Rate,!<si ) " b^trksi'S 

Corr(Rate,Ksi ) " r.trksi'; 

II II » II II • II M ■ 

"\f "; 

if (nJ == 0 or c_Df_c == 0) j 
goto last; 
end if ; 

Systematic Individual Differences";" "» 
crl t=seqa( 1 ? 1 snl ) ; 

Correlate -> " crl t' ; " " ; 

Cor r( Rate, W) " r.th.w'"; 

Beta(Rate.,W) " b_th_w'; 

Corr (Ksi.tOjW) " r_kstO_w=*; 

Beta(Ksi _tO,W) " b_kstO_w' ; 

t (Upper ) " t_u_s=' ; 

t( Lower) ' t_l_s';" "5" "; 

Correlation Matrix of X<t) <column5> and W <rows>";" "; 
Waves " tnr ; 

Correlate"; 
" " crl t'^r_;;„w; " ";" "j 

Correlation Matrix of Ksi't) <columns> and W <row5>";" "; 

Waves •-> " tm' ; 

Correlate"; 
" crlt~r_ksi_w; " ";" "; 

Regression Coefficients of X(t) <columns> on W <rowia:- " j " "; 

Waves -> " tm - , 

" Correlate"; 

" crlt'^b_>!_w; " ";" "; 
" Regression Coefficients of K5i(t) <columns> on W <row5>";" "; 

Waves -> " tm-; 

Correlate"; 
" crlt~b_.ksi_w; " " ; 
last : ; 
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